The (*Environment).Activate method only guarded against multiple calls
and therefore wasn't properly re-entrant.
If a plugin calls an UpdateConfiguration in the OnActivate hook, it would
again call this method, resulting in 2 plugin processes getting created.
To prevent that from happening, we pre-emptively set the status to running
just after the supervisor is created.
I am not sure why starting the plugin normally from the server does not
cause this because the root cause is the same and there should be 2 plugin
processes created in the normal scenario as well. I have not spent time
looking into that. But let me know if you want me to.
Unit test: This might be a bit hard to test via a unit test. I think
an e2e test maybe written in the playbooks test itself which counts
the number of processes that the server spawns.
https://mattermost.atlassian.net/browse/MM-44045
```release-note
NONE
```
* Fix PluginHealthCheckJob
We were incorrectly calling to start PluginHealthCheckJob
without initializing the plugins environment.
To fix this, we move the code inside initPlugins right after
the environment is initialized.
To respond to config changes, we call it again from pluginsEnvironment.Shutdown
which gets called from ConfigListener when plugins are disabled. And initPlugins
is anyways called again from ConfigListener which plugins are enabled,
so we can avoid checking for PluginSettings.Enable again in the call.
We also rename the method to better indicate its nature.
During this, we also uncover and fix another bug where disabling
plugins would not shut down plugins at all because we were calling
s.GetPluginsEnvironment() directly which returns nil if plugins
were disabled. The approach we follow is to manually acquire the lock
whenever we need access to the struct ignoring config setting.
We fix that as well.
https://community-daily.mattermost.com/boards/workspace/zyoahc9uapdn3xdptac6jb69ic/285b80a3-257d-41f6-8cf4-ed80ca9d92e5/495cdb4d-c13a-4992-8eb9-80cfee2819a4?c=6ef6178c-3512-4e57-8edd-1d2b66a09c9e
```release-note
NONE
```
* Fix test
```release-note
NONE
```
This PR builds up on the pass-through DB driver to a fully functioning DB driver implementation via our RPC layer.
To keep things separate from the plugin RPC API, and have the ability to move fast with changes, a separate field Driver is added to MattermostPlugin. Typically the field which is required to be compatible are the API and Helpers. It would be well-documented that Driver is purely for internal use by Mattermost plugins.
A new Driver interface was created which would have a client and server implementation. Every object (connection, statement, etc.) is created and added to a map on the server side. On the client side, the wrapper structs hold the object id, and communicate via the RPC API using this id.
When the server gets the object id, it picks up the appropriate object from its map and performs the operation, and sends back the data.
Some things that need to be handled are errors. Typical error types like pq.Error and mysql.MySQLError are registered with encoding/gob. But for error variables like sql.ErrNoRows, a special integer is encoded with the ErrorString struct. And on the cilent side, the integer is checked, and the appropriate error variable is returned.
Some pending things:
- Context support. This is tricky. Since context.Context is an interface, it's not possible to marshal it. We have to find a way to get the timeout value from the context and pass it.
- RowsColumnScanType(rowsID string, index int) reflect.Type API. Again, reflect.Type is an interface.
- Master/Replica API support.
* format using `goimports -local github.com/mattermost/mattermost-server/v5 -w`
* added goimports lint check to .golangci.yml
* format using `goimports -local github.com/mattermost/mattermost-server/v5 -w` for a corner case
* make app-layers, *-mocks and store-layers for ci check
Co-authored-by: Mahmudul Haque <mahmudulhaque@protonmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-31063: Change constants to use CamelCase
* store package
* change allcaps to camel case (#16615)
* New tools.mod
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* store failed timestamps on health check job instead of on registeredPlugin
Update test
* change EnsurePlugin calls
* Make env.SetPluginState private
* Write test for plugin deactivate and PluginStateFailedToStayRunning
* Add license comment
* adjust comments, use time.Since
* Additional PR feedback:
time.Since cleanup
test cleanup
remove duplicate .Store() call
* PR Feedback
- Add test case for reactivating the failed plugin
- Change `crashed` to `healthy` and `hasPluginCrashed` to `isPluginHealthy`
- remove stale timestamps from health check job
* Keep registeredPlugins in env when plugin is deactivated, so the crashed state of a plugin can be persisted.
* PR feedback
* PR feedback from Jesse
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-21019 - Fix race in (*Environment).SetPluginState()
- We change from passing pointers to registeredPlugin to passing
the struct by value.
- We also add a mutex to the supervisor struct to protect
from racy data access.
* move the immutability comment to the godoc of the Active method
* Changing mut to lock
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-19628: Fix race in (*registeredPlugin).State access
We used a pointer to an integer in the State variable, and therefore
setting a pointer value directly caused a race condition.
Since the registeredPlugin is a private struct, changing it to a normal int
creates fewer pointers and lets us to fix the race condition by simply setting
the integer value and then doing a registeredPlugins.Store.
The alternative to keeping the pointer would be to create a copy of each
member of a registeredPlugin and do a store again, which I feel is a roundabout
way to achieve this.
* Address review comments
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-17149 - Extend config.json for marketplace settings (#11933)
* MM-17149 - Extend config.json for marketplace settings
* Renamed MarketplaceUrl, tracking default marketplace url
* Added EnableMarketplace to the client config
* Revert "Added EnableMarketplace to the client config"
This reverts commit 0f982c4c661c2cd9bb96264e9a01a2363c40d9c5.
* MM-17149 - Added EnableMarketplace to the client config (#11958)
* Added EnableMarketplace to the client config
* Moved EnableMarketplace setting out of limited client configuration
* MM-17150, MM-17545, MM-18100 - Implement GET /api/v4/plugins/m… (#11977)
* MM-17150 - Implement GET /api/v4/plugins/marketplace proxying upstream
MM-17545 - Merge locally installed plugins into GET /api/v4/plugins/marketplace
* Replaced MarketplacePluginState with Installed
* Setting InstalledVersion instead of Installed
* marketplace client setting per_page if non zero
* Creating insecure client for marketplace url
* Fixed trailing slash for default marketplace url
* Adding filtering
* Fixed function names
* Renamed Manifest() to GetManifest(), added godoc for BaseMarketplacePlugin
* Handling plugin.ErrNotFound correctly
* Checking err == nil instead when a plugin is installed
* MM-18450 - Local-only plugin search (#12152)
* MM-17846: plugin icons (#12157)
* MM-17846: add support for plugin icons
Extend the model definitions to support plugin icons from the marketplace.
* s/IconURL/IconData
* MM-18475 - Converge on snake_case responses from the marketplace (#12179)
* MM-18520 - MM-Server should forward server version to marketplace server (#12181)
* Renamed request to filter client4.GetMarketplacePlugins
* Renamed request to filter
* Guarding against bad marketplace server response
* MM-17087 - Disable plugin on removal
* Updated documentation
* Got reid of notifyPluginEvents
* Updated documentation
* Added plugin installation/activatoin flow as a toplevel go doc in plugin_install.go
* Generating webapp bundle on plugin installation
* Fixed shadowing issue
* Updated doc to include unguarded race condition
* Renamed GenerateWebappBundle
* Added a debug log when peers are not ready to notify
* Updated docs
* Removed extra line
* Move State property from activePlugin to PluginHealthStatus. env.activePlugins is now reserved for healthy running plugins.
* Add comments for function declarations
* Combine activePlugins and pluginHealthStatuses into a common structure, registeredPlugins
* Add check to see if plugin is active before deactivating it
* Make `Deactivate` set plugin status
* Add comment explaining the `registeredPlugins` map
* Give responsibility to set plugin disabled status upon deactivation back to `env.Deactivate`
* check if plugin needs to be deactivated before setting status
* introduce plugin health check
* implement plugin health check job
* add support for checking pid of plugin process and RPC ping, to determine a plugin's health
* implement restart policy with back-offs
* support "EnableHealthCheck" boolean from config file.
* add tests for supervisor.PerformHealthCheck() and shouldDeactivatePlugin()
* improve error handling. clean up if blocks to be more concise
* MM-14575 - Automatically serve static files for plugins
* Added static handler for plugin public files
* Added StaticFilesPath method to Environment for use by MainRouter
* Added "static_files" property to Manifest Server
* Added unit tests for these changes
* MM-14575: Adding comment for cache control value
* MM-14575: Moved Static Plugin Request handler to plugin_requests
* Updated testing
* MM-14575: Removing the StaticFiles from Manifest Server
* MM-14575: Removing static files from test
* MM-14575: Updating static files test
* MM14575: Removing cache directive from plugin static files
* MM14575: Moving plugin public directory to root
* MM-14575: Updating tests for changed public directory
* MM-14575: Moved compileGo to a common utils package for tests
* MM-14575: Moving plugins initialization to InitPlugins find in tests
* Update utils/test_files_compiler.go
Adding Copyright header
Co-Authored-By: happygaijin <happygaijin@users.noreply.github.com>
* MM-14575: Consistent usage of static vs public name
* Removing spurious newline
* Comment typo
Co-Authored-By: happygaijin <happygaijin@users.noreply.github.com>
* Removing spurious new line
Co-Authored-By: happygaijin <happygaijin@users.noreply.github.com>
* MM14575: Adding a test to make sure only public files can be requested
* MM-14575 Adding a test for redirects on public files
* Fix shadowed variables in cmd package
* Fix shadowed variables in plugin package
* Fix shadowed variables in store package
* Fix shadowed variables in web package
* Changes as requested
Signed-off-by: Hanzei <hanzei@mailbox.org>
* Fix build
* Remove unnessary statements
* Use require all the time
* Fix build
* Rename variables according to feedback
* Fix NPE
* Changes as requested
* MM-7633: Optimize memory utilization during file uploads
Refactored the file upload code to reduce redundant buffering and stream
directly to the file store. Added tests.
Benchmark results:
```
levs-mbp:mattermost-server levb$ go test -v -run nothing -bench Upload -benchmem ./app
...
BenchmarkUploadFile/random-5Mb-gif-raw-ish_DoUploadFile-4 10 122598031 ns/op 21211370 B/op 1008 allocs/op
BenchmarkUploadFile/random-5Mb-gif-raw_UploadFileTask-4 100 20211926 ns/op 5678750 B/op 126 allocs/op
BenchmarkUploadFile/random-5Mb-gif-UploadFiles-4 2 1037051184 ns/op 81806360 B/op 3705013 allocs/op
BenchmarkUploadFile/random-5Mb-gif-UploadFileTask-4 2 933644431 ns/op 67015868 B/op 3704410 allocs/op
BenchmarkUploadFile/random-2Mb-jpg-raw-ish_DoUploadFile-4 100 13110509 ns/op 6032614 B/op 8052 allocs/op
BenchmarkUploadFile/random-2Mb-jpg-raw_UploadFileTask-4 100 10729867 ns/op 1738303 B/op 125 allocs/op
BenchmarkUploadFile/random-2Mb-jpg-UploadFiles-4 2 925274912 ns/op 70326352 B/op 3718856 allocs/op
BenchmarkUploadFile/random-2Mb-jpg-UploadFileTask-4 2 995033336 ns/op 58113796 B/op 3710943 allocs/op
BenchmarkUploadFile/zero-10Mb-raw-ish_DoUploadFile-4 30 50777211 ns/op 54791929 B/op 2714 allocs/op
BenchmarkUploadFile/zero-10Mb-raw_UploadFileTask-4 50 36387339 ns/op 10503920 B/op 126 allocs/op
BenchmarkUploadFile/zero-10Mb-UploadFiles-4 30 48657678 ns/op 54791948 B/op 2719 allocs/op
BenchmarkUploadFile/zero-10Mb-UploadFileTask-4 50 37506467 ns/op 31492060 B/op 131 allocs/op
...
```
https://mattermost.atlassian.net/browse/MM-7633https://github.com/mattermost/mattermost-server/issues/7801
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]
- [x] Added or updated unit tests (required for all new features)
- [ ] Added API documentation (required for all new APIs)
- [ ] All new/modified APIs include changes to the drivers
*N/A*???
- [x] Includes text changes and localization file ([.../i18n/en.json](https://github.com/mattermost/mattermost-server/blob/master/i18n/en.json)) updates
Overview of changes:
- api4
- Replaced `uploadFile` handler with `uploadFileStream` that reduces
unnecessary buffering.
- Added/refactored tests for the new API.
- Refactored apitestlib/Check...Status functions.
- app
- Added App.UploadFileTask, a more efficient refactor of UploadFile.
- Consistently set `FileInfo.HasPreviewImage`
- Added benchmarks for the new and prior implementations
- Replaced passing around `*image.Image` with `image.Image` in the
existing code.
- model
- Added a more capable `client4.UploadFiles` API to match the new server
API’s capabilities.
- I18n
- Replaced `api.file.upload_file.bad_parse.app_error` with a more generic
`api.file.upload_file.read_request.app_error`
- plugin
- Removed type `plugin.multiPluginHookRunnerFunc` in favor of using
`func(hooks Hooks) bool` explicitly, to help with testing
- tests
- Added test files for testing images
Still remaining, but can be separate PRs - please let me know the preferred
course of action
- Investigate JS client API - how does it do multipart?
- Performance loss from old code on (small) image processing?
- Deprecate the old functions, change other API implementations to use
UploadFileTask
Definitely separate future PRs - should I file tickets foe these?
- Only invoke t.readAll() if there are indeed applicable plugins to run
- Find a way to leverage goexif buffer rather than re-reading
Suggested long-term improvements - should I file separate tickets for these?
- Actually allow uploading of large (GB-sized) files. This may require a
change in how the file is passed to plugins.
- (Many) api4 tests should probably be subtests and share a server setup -
will be much faster
- Performance improvements in image processing (goexif/thumbnail/preview)
(maybe use https://mattermost.atlassian.net/browse/MM-10188 for this)
Questions:
1. I am commiting MBs of test images, are there better alternatives? I can
probably create much less dense images that would take up considerably less
space, even at pretty large sizes
2. I18n: Do I need to do anything special for the string change? Or just wait
until it gets picked up and translated/updated?
3. The image dimensions are flipped in resulting FileInfo to match the actual
orientation. Is this by design? Should add a test for it, perhaps?
4. What to do in the case of partial success? S3 but not DB, some files but not
others? For now, just doing what the old code did, I think.
5. Make maxUploadDrainBytes configurable? Also, should this be the systemic
behavior of all APIs with non-empty body? Otherwise dropped altogether?
Check all other ioutil.ReadAll() from sockets. Find a way to set a total
byte limit on request Body?
* WIP - Fixed for GetPluginsEnvironment() changes
* WIP - PR feedback
1. Refactored checkHTTPStatus to improve failure messages
2. Use `var name []type` rather than `name := ([]type)(nil)`
3. Replaced single-letter `p` with a more intention-revealing `part`
4. Added tests for full image size, `HasPreviewImage`
* WIP - rebased (c.Session->c.App.Session)
* WIP - PR feedback: eliminated use of Request.MultipartReader
Instead of hacking the request object to use r.MultipartReader now have own
functions `parseMultipartRequestHeader` and `multipartReader` eliminating the
need to hack the request object to use Request.MultipartReader limitations.
* WIP - PR feedback: UploadFileX with functional options
* WIP - PR feedback: style
* WIP - PR feedback: errors cleanup
* WIP - clarified File Upload benchmarks
* WIP - PR feedback: display the value of erroneous formname
* WIP - PR feedback: fixed handling of multiple channel_ids
* WIP - rebased from master - fixed tests
* PR Feedback
* PR feedback - moved client4.UploadFiles to _test for now
Fix code issues in channel_test.go
Fix Channel Test Issues detected by Megacheck
Fix API Emoji Test Issues detected by Megacheck
Fixed API Issues Reported by Megacheck
Fixed App issues reported by megacheck
Remaining fixes
removed test added by mistake from old HEAD
gofmt
Store Fixes
simplified returns
Fix test for multi member channel delete
revert to delete unused function
* plugins: compute bundle hash on load
Use this hash to bust client caches whenever the plugin bundle changes.
* eliminate redundant pluginHandler
* switch to 64-bit FNV-1a
* Fix test
* ensure plugin is always shutdown
Once we call `.client.Client()` the plugin has started, and must be shut
down. `newSupervisor` sometimes returned with an error (and without a
reference to the supervisor), leaving the client running indefinitely.
* Clarify the documentation to explain that plugin hooks will not trigger until `OnActivate` returns successfully, and will stop triggering just before `OnDeactivate` is called.
* test for plugin deadlock
* plugin/environment.go: switch to sync.Map
From: https://golang.org/pkg/sync/#Map
> If a goroutine holds a RWMutex for reading and another goroutine might call Lock, no goroutine should expect to be able to acquire a read lock until the initial read lock is released. In particular, this prohibits recursive read locking. This is to ensure that the lock eventually becomes available; a blocked Lock call excludes new readers from acquiring the lock.
The previous `RWMutex` was not safe given that we effectively acquired read locks recursively (hook -> api -> hook). This worked up until we activated or deactivated plugins, tried to acquire a write lock, and the plugin used the API to effectively trigger another hook.
Switching to sync.Map avoids this by divesting the need to lock at all, avoiding the potential for a recursive lock in the first place.
* Support `server`, deprecate `backend` in plugin manifest
This lets us converge on the use of the term `server` everywhere instead
of sometimes `backend` and sometimes `server`. We're still using
`webapp` and will eventually support `mobile` as well.
The plan is actually to rip out these deprecations as part of releasing
5.2, but I want to coordinate the extra additional breakage at the same
time, so for now this is a backwards compatible change.
* fix failing tests
* clean up plugins GoDoc:
- eliminate plugin.NewBlankContext() as unnecessary
- export ValidIdRegex as a string vs. the less readable var
- add/update various documentation strings
- hide everything by default, except where used by client plugins or the mattermost-server. The exception to this rule are the `*(Args|Returns)` structs which must be public for go-plugin, but are now prefixed with `Z_` with a warning not to use.
- include a top-level example to get plugin authors started
This is not a breaking change for existing plugins compiled against
plugins-v2.
* remove commented out ServeHTTPResponseWriter
* update examples to match developer docs
* add missing plugin/doc.go license header
