* Don't downgrade plugins with feature flag for on-prem
* Update app/plugin.go
Co-authored-by: Christopher Poile <cpoile@gmail.com>
* Short circut for plugin doesn't exist.
Co-authored-by: Christopher Poile <cpoile@gmail.com>
* format using `goimports -local github.com/mattermost/mattermost-server/v5 -w`
* added goimports lint check to .golangci.yml
* format using `goimports -local github.com/mattermost/mattermost-server/v5 -w` for a corner case
* make app-layers, *-mocks and store-layers for ci check
Co-authored-by: Mahmudul Haque <mahmudulhaque@protonmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Installing plugins specified by feature flags using the marketplace.
* Switch back to using getplugins client.
* Respect disabling automatic installation of pluings.
* pluginid -> plugin_id
* Debug logs for enable plugin error
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Start plugins concurrently
* Determine which plugins need to be activated/deactivated before changing their state
* Test activation/deactivation of plugins in TestSyncPluginsActiveState
* Change test comments
* remove unneeded temp var
* Removing some other fake apps
* More FakeApp removed
* Removing entirely FakeApp
* Fixing some tests
* Fixing get Cluster id from get plugin status
* Fixing failing tests
* Fixing tests
* Fixing test initialization for web
* Fixing InitServer for server tests
* Fixing InitServer for server tests
* Reverting go.sum and go.mod
* Removing unneded HTMLTemplates function in App layer
* Moving back some functions to its old place to easy the review
* Moving back some functions to its old place to easy the review
* Using the last struct2interface version
* Generating store layers
* Fixing merge problems
* Addressing PR comments
* Small fix
* Fixing app tests build
* Fixing tests
* fixing tests
* Fix tests
* Fixing tests
* Fixing tests
* Fixing tests
* Moving license to server struct
* Adding some fixes to the test compilation
* Fixing cluster and some jobs initialization
* Fixing some license tests compilation problems
* Fixing recursive cache invalidation
* Regenerating app layers
* Fix test compilation
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
There was a race in (*App).SyncPlugins where if the same plugin
existed in availablePlugins and pluginSignaturePathMap, then
we would try to add/remove at the same time.
This would lead to a possible removal / addition of a plugin directory
or even unable to remove a directory because it was already in use.
We fix this by first finishing the removal of availablePlugins
before syncing it with the file store.
* [MM-20941] Parallelize plugin loading on startup
* [MM-20941] Parallelize plugin SyncPlugins
* [MM-20941] Make activation and deactivation of plugin concurrent
* [MM-20941] Remove uncessary optimization
This reverts commit aad2680aec843125bf72d8d2406869f4948dba42.
[MM-20941] Remove uncessary wait in SyncPlugins
[MM-20941] Parallelize processing of prepackaged plugins
* [MM-20941] Fix lint issue
* [MM-20941] Fix bug for getting the number of plugins to process
* [MM-20941] Use buffered channel and fix return issue in prepackaged plugin processing
* [MM-20941] Implement feedback
* [MM-20941] Fix looping over channel for plugins
* [MM-20941] Retain signature of process plugin
* [MM-20941] Fix function documentation
* [MM-20941] Fix bug to pass the plugin path in anonymous function
* [MM-20941] Rename variable
* [MM-20941] Make minor changes in variable naming
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-21103: change plugin signature path
Save as `<plugin_id>.tar.gz.sig` instead of `<plugin_id>.sig`. The latter was a relic of the previous design to support multiple plugin signatures, but now creates an inconsistency with how the original source files were supplied as `<some_name>.tar.gz` and `<some_name>.tar.gz.sig`.
Fixes: https://mattermost.atlassian.net/browse/MM-21103
* relax signature matches to avoid assuming signatures always exist
* Consistent license message for all the go files
* Fixing the last set of unconsistencies with the license headers
* Addressing PR review comments
* Fixing busy.go and busy_test.go license header
* MM-17549: use StatusNotFound when deleting plugin (#12983)
Trying to delete a plugin that does not exist should fail with a 404, not a 400.
Relates-to: https://mattermost.atlassian.net/browse/MM-17549
* MM-19630: marketplace: model ReleaseNotesURL (#13083)
Automatic Merge
* MM-20065: allow prepackaged plugin upgrade (#13076)
* MM-20065: allow prepackaged plugin upgrade
When locally installing prepackaged plugins, skip if a plugin exists with the same id and is the same or a newer version.
This is effectively a "poor man's" rework of prepackaged plugins to allow upgrade of prepackaged plugins via the marketplace. The larger plan to rework prepackaged plugins was deferred from v5.18.
Fixes: https://mattermost.atlassian.net/browse/MM-20065
* eliminate unnecessary installPlugin
* fix TestPluginSync defaults to match minio
* cleanExistingBundles
* close prepackaged filereader
* simplify
* MM-18540 - Demo plugin crashes on server shutdown
* MM-18540 - Removed unncessary lock statements
* Added comments, logging a warning if PluginsEnv is updated while shutting down plugins
* Don't clean up PluginsEnv if another Env is detected
* Changing warn to debug
* Revert "Changing warn to debug"
This reverts commit 46f20ab21eeabb01d07f53e02d850cd6d9b2b837.
* MM-17149 - Extend config.json for marketplace settings (#11933)
* MM-17149 - Extend config.json for marketplace settings
* Renamed MarketplaceUrl, tracking default marketplace url
* Added EnableMarketplace to the client config
* Revert "Added EnableMarketplace to the client config"
This reverts commit 0f982c4c661c2cd9bb96264e9a01a2363c40d9c5.
* MM-17149 - Added EnableMarketplace to the client config (#11958)
* Added EnableMarketplace to the client config
* Moved EnableMarketplace setting out of limited client configuration
* MM-17150, MM-17545, MM-18100 - Implement GET /api/v4/plugins/m… (#11977)
* MM-17150 - Implement GET /api/v4/plugins/marketplace proxying upstream
MM-17545 - Merge locally installed plugins into GET /api/v4/plugins/marketplace
* Replaced MarketplacePluginState with Installed
* Setting InstalledVersion instead of Installed
* marketplace client setting per_page if non zero
* Creating insecure client for marketplace url
* Fixed trailing slash for default marketplace url
* Adding filtering
* Fixed function names
* Renamed Manifest() to GetManifest(), added godoc for BaseMarketplacePlugin
* Handling plugin.ErrNotFound correctly
* Checking err == nil instead when a plugin is installed
* MM-18450 - Local-only plugin search (#12152)
* MM-17846: plugin icons (#12157)
* MM-17846: add support for plugin icons
Extend the model definitions to support plugin icons from the marketplace.
* s/IconURL/IconData
* MM-18475 - Converge on snake_case responses from the marketplace (#12179)
* MM-18520 - MM-Server should forward server version to marketplace server (#12181)
* Renamed request to filter client4.GetMarketplacePlugins
* Renamed request to filter
* Guarding against bad marketplace server response
* MM-17087 - Disable plugin on removal
* Updated documentation
* Got reid of notifyPluginEvents
* Updated documentation
* Added plugin installation/activatoin flow as a toplevel go doc in plugin_install.go
* Generating webapp bundle on plugin installation
* Fixed shadowing issue
* Updated doc to include unguarded race condition
* Renamed GenerateWebappBundle
* Added a debug log when peers are not ready to notify
* Updated docs
* Removed extra line
* MM-16272 - Synchronize plugins across cluster (#11611)
* MM-16272 - Synchronize plugins across cluster
* Adding a test
* MM-16272 - Fixed tests
* MM-16272 - PR feedback
* MM-16270 - Plugin Sync (#11615)
* Initial implementation for plugin synch with file store. WIP
* Removed ListAll implementation. Used ListDirectory and change localstore to be consistent and return all items (files and folders) from directory
* Refactored plugin filestore operations out of main install/remove plugin
* Fixing error handling details
* Changes to use structured logging
* More logging fixes
* Wording and comments improvements
* Error handling and control flow improvements
* Changed managed flag check to use os.stat
* Added file store plugin dir and filename consts
* Replaced FileRead to use a the FileReader in PluginSync
* Minor styling and PR feedback changes
* Minor error handling improvements
* Added unit test for SyncPlugins. Changed SyncPlugins to use plugins environment to list available plugins
* PR Feedback improvements
* Minor err handling fix
* Removing FileStorePath from PluginEventData (#11644)
* Fix plugin path (#11654)
* tweak path, logging
Fix an issue not finding the plugins folder in S3. Tweak logging messages to add additional clarity.
* Removing FileExists check when Syncing plugins. Updated localstore to not return an error when directory does not exist
* PR Feedback
* Install prepackaged plugins locally only (#11656)
* s/uninstall/remove
* Updated ClusterMessage comment
* Updated PluginSync to test against s3 + local storage
* guard app plugins with mutex
Shutting down the app could race with a goroutine that uses the plugins environment, since we shut down the plugins first before cleaning up goroutines.
* fix go vet issues
* Moving goroutine pool
* Auto refactor
* Moving plugins.
* Auto refactor
* Moving fields to server
* Auto refactor
* Removing siteurl duplication.
* Moving reset of app fields
* Auto refactor
* Formatting
* Moving niling of Server to after last use
* Fixing unit tests.
Expire K/V Values
Regenerate Code
pathfix
Update Expiry on Update
Check for Exit Signal
gofmt
Rewrote Go Routine
Remove tempoarily cleanup loop
fix expiretime
TEST: Expired Watchdog as GoRoutine
Check if Srv is nil
Use Scheduler/Worker for Expired Key CleanUp
add license
fix scheduler job type; DoJob Restructuring
Remove unused imports and constants
move db migration from 5.4 to 5.5
* plugins: compute bundle hash on load
Use this hash to bust client caches whenever the plugin bundle changes.
* eliminate redundant pluginHandler
* switch to 64-bit FNV-1a
* Fix test
* allow `Wait()`ing on the supervisor
In the event the plugin supervisor shuts down a plugin for crashing too
many times, the new `Wait()` interface allows the `ActivatePlugin` to
accept a callback function to trigger when `supervisor.Wait()` returns.
If the supervisor shuts down normally, this callback is invoked with
a nil error, otherwise any error reported by the supervisor is passed
along.
* improve plugin activation/deactivation logic
Avoid triggering activation of previously failed-to-start plugins just
becase something in the configuration changed. Now, intelligently
compare the global enable bit as well as the each individual plugin's
enabled bit.
* expose store to manipulate PluginStatuses
* expose API to fetch plugin statuses
* keep track of whether or not plugin sandboxing is supported
* transition plugin statuses
* restore error on plugin activation if already active
* don't initialize test plugins until successfully loaded
* emit websocket events when plugin statuses change
* skip pruning if already initialized
* MM-8622: maintain plugin statuses in memory
Switch away from persisting plugin statuses to the database, and
maintain in memory instead. This will be followed by a cluster interface
to query the in-memory status of plugin statuses from all cluster nodes.
At the same time, rename `cluster_discovery_id` on the `PluginStatus`
model object to `cluster_id`.
* MM-8622: aggregate plugin statuses across cluster
* fetch cluster plugin statuses when emitting websocket notification
* address unit test fixes after rebasing
* relax (poor) racey unit test re: supervisor.Wait()
* make store-mocks
* don't report an error on plugin activation if already active
* improved plugin logging events
Log an error when a plugin's ServeHTTP fails, or when it unexpectedly
terminates.
Restart a plugin at most three times, allowing its failure to later
bubble up under the "failed to stay running" status.
* clarified plugin activation/deactivation
Avoid repeatedly activating when any configuration bit changes. Improved
logging.
* constrain plugin ids to ^[a-zA-Z0-9-_\.]+$ and enforce minimum length
Previously, the plugin id was used unsanitized to relocate the plugin
bundle, which allowed writing outside the `plugins/` directory by using
an `id` containing `../`.
Similarly, an empty string was accepted as an id and led to unexpected
error messages.
* remove plugins by manifest path, not id
If the id within the manifest ever diverges from the actual plugin
location, it becomes impossible to remove via the API. Instead, if the
plugin is found by id, remove the path containing the manifest.
* ignore plugins with nil manifests
If a plugin was detected, but had a manifest that couldn't be parsed, it
will be left nil but still be listed among the packages. Skip over these
in most cases to avoid segfaults.
* leverage mlog more effectively for plugins
* build issues
