https://mattermost.atlassian.net/browse/MM-22051
```release-note
Removed the following methods/functions:
(ad *AccessData) ToJson()
(ar *AccessResponse) ToJson()
(ar *AnalyticsRow) ToJson()
(ar AnalyticsRows) ToJson()
(o *Audit) ToJson()
(o Audits) ToJson()
(ad *AuthData) ToJson()
(ar *AuthorizeRequest) ToJson()
(o *ChannelPatch) ToJson()
(o *ChannelsWithCount) ToJson()
(o *ChannelCounts) ToJson()
(o *ChannelData) ToJson()
(o *ChannelMembers) ToJson()
(o *ChannelUnread) ToJson()
(o *ChannelUnreadAt) ToJson()
(o *ChannelStats) ToJson()
(o *ChannelView) ToJson()
(o *ChannelViewResponse) ToJson()
(o *ClusterDiscovery) ToJson()
(ci *ClusterInfo) ToJson()
(cs *ClusterStats) ToJson()
(o *Command) ToJson()
CommandListToJson(l []*Command) string
(o *CommandArgs) ToJson()
(cmr *CommandMoveRequest) ToJson()
(o *CommandResponse) ToJson()
(c *Compliance) ToJson()
(c Compliances) ToJson()
(o *Config) ToJson()
EmojiListToJson(emojiList []*Emoji)
```
* MM-25516: Changed to byte slice instead of string for cluster messages
https://mattermost.atlassian.net/browse/MM-25116
Testing:
Manually tested.
Load-tested with Cluster Controller.
I looked into changing the serialization method to use msgpack,
but the ClusterMessage struct was mainly used for only 3 fields
which didn't lead to much of a CPU time improvement, whereas
actually led to more allocations using msgpack. Hence, I chose
to remain with JSON.
```
name old time/op new time/op delta
ClusterMarshal-8 3.51µs ± 1% 3.10µs ± 2% -11.59% (p=0.000 n=9+10)
name old alloc/op new alloc/op delta
ClusterMarshal-8 776B ± 0% 1000B ± 0% +28.87% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
ClusterMarshal-8 12.0 ± 0% 13.0 ± 0% +8.33% (p=0.000 n=10+10)
```
```release-note
Changed the field type of Data in model.ClusterMessage to []byte from string.
```
* Trigger CI
```release-note
NONE
```
We replace the double conversion of
[]byte to string, with a direct write
to http.ResponseWriter.
https://mattermost.atlassian.net/browse/MM-22051
Tried using gofmt -r, but it only accepts Go
expressions. So had to resort to an ugly sed replace
sed -E -i 's/w.Write\(\[\]byte\((.*).ToJson\(\)\)\)/if err := json.NewEncoder\(w\).Encode\(\1\); err != nil { mlog.Warn\("Error while writing response", mlog.Err\(err\)\)}/g' *.go
```release-note
NONE
```
* Cleanup model/utils.go
- Removed some functions which were unsued. (The unused linter was somehow failing to catch this).
- Moved some functions inside other packages.
- Removed two duplicate instances of the same function - RemoveDuplicateStrings and UniqueStrings.
- Moved some regexes to global vars to prevent compilation every time.
```release-note
NONE
```
* fix lint
```release-note
NONE
```
* bring back unused exception
```release-note
NONE
```
* fix tests
```release-note
NONE
```
* Address review comments
```release-note
NONE
```
* MM-21357: Use typed constant for channel types
https://mattermost.atlassian.net/browse/MM-21357
```release-note
- Introduced a new type ChannelType for all channel types.
- Updated the Client4.UpdateChannelPrivacy method to ChannelType.
```
* Address review comments
```release-note
NONE
```
* telemetry fix
```release-note
NONE
```
* MM-34434: Added 'is_trial' boolean to all trial license requests and to the License struct.
* MM-34434: Generalized the concept of a license request.
* MM-34434: Verifies JSON field of license instance is set.
* MM-34434: Added missing client param.
* MM-34434: Added some tests of the request trial API endpoint.
* MM-34434: Removed comment.
* fix broken test (#17348)
* Add missing wrapped errors (#17339)
* Improve document extraction and including a document extraction command (#17183)
* Add extract documents content command
* Adding the extraction command and making the pure go pdf library as secondary option
* Improving the memory usage and docextractor interface
* Enable content extraction by default in all the instances
* Tiny improvement on archive indexing
* Adding App interface generation and the opentracing layer
* Fixing linter errors
* Addressing PR review comments
* Addressing PR review comments
* Update en.json (#17356)
Automatic Merge
* adding new feature flag (#17308)
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Bump no_output_timeout to 2 hours (#17358)
* log invalid username (#17345)
Automatic Merge
* MM-34434: Added missing client param.
MM-34434: Added some tests of the request trial API endpoint.
MM-34434: Removed comment.
* MM-34434: Switched to a hard-coded true value.
* MM-34434: Reverts test change.
* MM-34434: Removes unnecessary field.
* MM-34434: Tests that is_trial is hard-coded by TrialLicenseRequest.
* MM-34434: Removed accidental commit.
* MM-34434: Removes unnecessary is_trial key from JSON payload.
* MM-34434: Reverts to old pointer receiver variable name.
* MM-34434: Removes test.
* #MM-34437 Initialized license service
* ##MM-34437 Verified at all points if server is trial elligible
* WIp
* #MM-34437 removed unused commented code
* MM-34437 make a log less severe
* #MM-34437 generated einterface mocks
* #MM-34437 added license on new file
* #MM-34437 removed unused translation
* #MM-34437 some refactoring
* Update api4/license.go
* Update api4/license.go
* #MM-34437 made a variable name consistent
* #MM-34437 Added mocks for lince validator
* #M--34437 Added license validator test framework
* #MM-34437 Renamed isTrial method to isTrialLicense to avoid conflict with newlya dded field
* #M--34437 Allowed sales-sanctioned trials
* #MM-34437 fixed trial license API tests
* Added tests for add license API
* #MM-34437 fixed ValidateLicense test
* #MM-34437 Added util tests
* #MM-34437 using NoError for checking no error
* #MM-34437 using NoError for checking no error
* Added dummy piblic key for testing
* Fixed tests
* #MM-34437 udpaetd trial license URL for testing
* #MM-34437 adjusted times for licences generated through admin portal
* Reverted test-only changes
Co-authored-by: Martin Kraft <martin@upspin.org>
Co-authored-by: Hossein <hahmadia@users.noreply.github.com>
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
Co-authored-by: Jesús Espino <jespinog@gmail.com>
Co-authored-by: Amy Blais <amy_blais@hotmail.com>
Co-authored-by: Ben Cooke <benkcooke@gmail.com>
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
Co-authored-by: Max Erenberg <max.erenberg@mattermost.com>
* Revert "Revert "[MM-8497] Ability to set Do Not Disturb for a specified period of time (#16067)" (#17657)"
This reverts commit ff383990f8.
* add debug log for recurring function
* add feature flag for dnd timed status
* refactoring changes
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Update store function GetThreadForUser to use master DB to fix replica lag
* Refactor GetThreadForUser store func to take membership as argument to prevent replica lag issues and reduce joins in query
* Add translation
* Fix test
* Updates per feedback
* Minor clean-up per feedback
* MM-34434: Added 'is_trial' boolean to all trial license requests and to the License struct.
* MM-34434: Generalized the concept of a license request.
* MM-34434: Verifies JSON field of license instance is set.
* MM-34434: Added missing client param.
* MM-34434: Added some tests of the request trial API endpoint.
* MM-34434: Removed comment.
* fix broken test (#17348)
* Add missing wrapped errors (#17339)
* Improve document extraction and including a document extraction command (#17183)
* Add extract documents content command
* Adding the extraction command and making the pure go pdf library as secondary option
* Improving the memory usage and docextractor interface
* Enable content extraction by default in all the instances
* Tiny improvement on archive indexing
* Adding App interface generation and the opentracing layer
* Fixing linter errors
* Addressing PR review comments
* Addressing PR review comments
* Update en.json (#17356)
Automatic Merge
* adding new feature flag (#17308)
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Bump no_output_timeout to 2 hours (#17358)
* log invalid username (#17345)
Automatic Merge
* MM-34434: Added missing client param.
MM-34434: Added some tests of the request trial API endpoint.
MM-34434: Removed comment.
* MM-34434: Switched to a hard-coded true value.
* MM-34434: Reverts test change.
* MM-34434: Removes unnecessary field.
* MM-34434: Tests that is_trial is hard-coded by TrialLicenseRequest.
* MM-34434: Removed accidental commit.
* MM-34434: Removes unnecessary is_trial key from JSON payload.
* MM-34434: Reverts to old pointer receiver variable name.
* MM-34434: Removes test.
Co-authored-by: Hossein <hahmadia@users.noreply.github.com>
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
Co-authored-by: Jesús Espino <jespinog@gmail.com>
Co-authored-by: Amy Blais <amy_blais@hotmail.com>
Co-authored-by: Ben Cooke <benkcooke@gmail.com>
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
Co-authored-by: Max Erenberg <max.erenberg@mattermost.com>
* Added expiry support in custom status APIs (#11)
* Added expiry support in custom status APIs
Added validation for the duration and expiration time in request body
Made enum for the custom status durations
* Fixed the bug in expiry validation with dont clear validation
* Fixed review comments
Converted the durations enum to map
Removed extra if-else
* Added expiry support in custom status slash command (#17)
* Added expiry support in custom status slash command
* Added the check for timezone enabled in expiry time in custom status slash command
* Review fixes
Changed name of calculateExpriryTime to calculateEndOfDay
Made function SetDefaultEmoji for settting default emoji in set custom status API
* Added support for empty duration in custom status APIs
Made one of emoji and text required and duration optional in set custom status API
Made default duration dont clear in both API and slash command
* Changed value of ExpiresAt field in custom status slash command
* Code refactoring
Combined SetDefaults and TrimMessage into 1 function PreSave
Refactored isExpirationTimeValid function
* Used model variables instead of new variables in custom status slash command
* Modified behaviour of set custom status APIs (#19)
Removed dont_clear from validCustomStatusDuration map
Added logic to set duration custom date/time if only expires_at is specified in the body
Made function AreDurationAndExpirationTimeValid in custom status model
* Trigger CI build
* Add a controller for changing subscription
* Changes for PR
* Another change
* Fix pipeline
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Replace config generator
* Cleanup
* Some renaming and docs additions to add clarity
* Cleanup logging related methods
* Cleanup emitter
* Fix TestDefaultsGenerator
* Move feature flags synchronization logic out of config package
* Remove unnecessary util functions
* Simplify load/set logic
* Refine semantics and add some test to cover them
* Remove unnecessary deep copies
* Improve logic further
* Fix license header
* Review file store tests
* Fix test
* Fix test
* Avoid additional write during initialization
* More consistent naming
* Update app/feature_flags.go
Co-authored-by: Christopher Speller <crspeller@gmail.com>
* Update config/store.go
Co-authored-by: Christopher Speller <crspeller@gmail.com>
* Update config/store.go
Co-authored-by: Christopher Speller <crspeller@gmail.com>
* Update config/store.go
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* Make ConfigStore.Set() return both old and new configs
* Implement config diff function
* Make app.SaveConfig return previous and current configs
* Add config diff to audit record
* Fix returned configs
* Include high level test
* Move FF synchronizer to its own package
* Remove unidiomatic use of sync.Once
* Add some comments
* Rename function
* More comment
* Save config diff in audit record for local endpoints
* Enable audit for config set/reset commands
* Improve tests output
Co-authored-by: Christopher Speller <crspeller@gmail.com>
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* restrict creation of direct channels to team members
* run make i18n-extract
* add suggestions from hahmadia
* place common-team-check logic in app layer
* use flat SQL query
* show more specific error message to user
* MM-7968: Fmt file.
* MM-7968: Fix for moved session field.
Co-authored-by: Max Erenberg <max.erenberg@mattermost.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Martin Kraft <martinkraft@gmail.com>
Co-authored-by: Martin Kraft <martin@upspin.org>
* Replace config generator
* Cleanup
* Some renaming and docs additions to add clarity
* Cleanup logging related methods
* Cleanup emitter
* Fix TestDefaultsGenerator
* Move feature flags synchronization logic out of config package
* Remove unnecessary util functions
* Simplify load/set logic
* Refine semantics and add some test to cover them
* Remove unnecessary deep copies
* Improve logic further
* Fix license header
* Review file store tests
* Fix test
* Fix test
* Avoid additional write during initialization
* More consistent naming
* Update app/feature_flags.go
Co-authored-by: Christopher Speller <crspeller@gmail.com>
* Update config/store.go
Co-authored-by: Christopher Speller <crspeller@gmail.com>
* Update config/store.go
Co-authored-by: Christopher Speller <crspeller@gmail.com>
* Update config/store.go
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* Move FF synchronizer to its own package
* Remove unidiomatic use of sync.Once
* Add some comments
* Rename function
* More comment
Co-authored-by: Christopher Speller <crspeller@gmail.com>
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* add request context
* move initialialization to server
* use app interface instead of global app functions
* remove app context from webconn
* cleanup
* remove duplicated services
* move context to separate package
* remove finalize init method and move content to NewServer function
* restart workers and schedulers after adding license for tests
* reflect review comments
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Add support for timed DND status
- accept a date time value in api query when dnd mode for user needs to be unset
- Create a new function to handle SetDNDStatus calls
- Create a scheduled task to unset dnd mode to wahtever mode was before setting it to DND
* update schema version
* Model changes to make fields more intuitive
- move dndendtime to status model
- add new field prev status in status to keep track of previous status of user
- update db migration function
- make use of prevstatus and dndendtime from status model
* set prev status and dndendtime appropriately after unsetting dnd mode
* add json tag for dndendtime
* unset dnd status only if not changed manually by user
* update dnd statuses after server restart
* make app-layers
* fix failing tests
* don't create sched task when setting status to DND
* get only expired statuses from db
- convert end time from any timezone to utc
- store dnd end time in unix format for usability reasons
* run update dnd status only on leader
* make mocks
* fix tests
* run UpdateDNDStatusOfUsers as recurring task
* save all statuses at once in db and update UpdateDNDStatusOfUsers logic
* add app method to get timezone of user
* store dnd end time in context.Params
* set max size of prevstatus
* update status model to take endtime input as string and store in db as unix time(int64)
* Add tests for SetStatusDoNotDisturbTimed
* if dnd_end_time is not passed the call old api to set dnd mode
* fix tests
* new plugin api to use new timed dnd mode
* get and update rows in a single db query
* dnd end time will be stored in request body and not route param
* exclude statuses which has dndendtimeunix < 0
* update and get the updated dnd statuses in single db query
* add updated status to cache
* DNDEndTimeUnix and PrevStatus need not to be visible to users
* update db schema version for migration
* Keep Status and PrevStatus varchar size same
* add test to verify status is restored after dnd end time expires
* expect endtime in utc from client
- remove store method GetTimezone as no longer needed
- add documentation for SetStatusDoNotDisturbTimed
* reduce sleep time for dnd timed restore test
* more appropriate name for new api to update user status
* update db migration function
* parse and validate time before potentially triggering db query to get status of user
* add migration changes in to existing upgrade function
* not supporting un-timed dnd status via api
* don't call Srv.Store directly, call via app layer
* rename dndendtime to statuscleartime to make it suitable for custom status usage as well
* Revert "rename dndendtime to statuscleartime to make it suitable for custom status usage as well"
This reverts commit fa69152d9a3db18f1c59b34c878fb7ce494440b5.
* mysql doesn't support RETURNING clause so add tx to get and update statuses
* add UpdateDNDStatusOfUsers mock in tests
* update store mock import path
* add mock in storelib
* Add status mocks to empty store
* Close the task during server shutdown
* Do not cancel a nil task
* update squirrel queries
* remove untimed dnd test
* start recurring task to unset statuses on leadership change
* set dndTask to nil after cancelling it upon server shutdown
* new recurring task which starts at nearest rounded time of the interval
* mock Get() call for status
* return updated statuses in case of mysql
* remove unneccessary code
* add Get() mock to empty store
* fix mocking for once and all
* address review comments
fix mysql updateStatus fn
protect dndTask with mutex
minor refactors
* move runDNDStatusExpireJob to server.go and pass App as arg instead of method receiver
* frontend will send endtime in unix epoch format so get rid of double representation
* scan for all fields and not just two
* add some tests and fix review comments
* remove extra sql query and create needed result in go
* add storetest for UpdateExpiredDNDStatuses
* add migrations to latest version
* update min supported version
* add comment to fix a bug in future
* update test to expect 1 status in return
* rename UpdateUserStatusWithDNDTimeout to SetUserStatusTimedDND
* rename DNDEndTimeUnix to DNDEndTime
* cast int to int64 for equality
* fix tests and error handling
* move updating values to retrieved statuses fields outside sql transaction
* move migrations to 5.36
Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* add includeRemovedMembers flag
* fix API call in client4.go
* remove check for 'since'
* add comments
* run make app-layers
* re-run CI tests
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Removing FilesSearch feature flag
* Fixing tests
* Adding an improvement on plain text extraction
* Adding tests for plain text extraction
* Removed unneeded conversion
* Adding missed license
* Remove the feature flag from the migration
* Fixing some tests
* Updating i18n/en.json file
* MM-33601: Invalidate team member cache when updating a team scheme.
* MM-33601: Tests cache invalidation.
* MM-33601: Test fix.
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
