* restrict creation of direct channels to team members
* run make i18n-extract
* add suggestions from hahmadia
* place common-team-check logic in app layer
* use flat SQL query
* show more specific error message to user
* MM-7968: Fmt file.
* MM-7968: Fix for moved session field.
Co-authored-by: Max Erenberg <max.erenberg@mattermost.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Martin Kraft <martinkraft@gmail.com>
Co-authored-by: Martin Kraft <martin@upspin.org>
* Replace config generator
* Cleanup
* Some renaming and docs additions to add clarity
* Cleanup logging related methods
* Cleanup emitter
* Fix TestDefaultsGenerator
* Move feature flags synchronization logic out of config package
* Remove unnecessary util functions
* Simplify load/set logic
* Refine semantics and add some test to cover them
* Remove unnecessary deep copies
* Improve logic further
* Fix license header
* Review file store tests
* Fix test
* Fix test
* Avoid additional write during initialization
* More consistent naming
* Update app/feature_flags.go
Co-authored-by: Christopher Speller <crspeller@gmail.com>
* Update config/store.go
Co-authored-by: Christopher Speller <crspeller@gmail.com>
* Update config/store.go
Co-authored-by: Christopher Speller <crspeller@gmail.com>
* Update config/store.go
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* Move FF synchronizer to its own package
* Remove unidiomatic use of sync.Once
* Add some comments
* Rename function
* More comment
Co-authored-by: Christopher Speller <crspeller@gmail.com>
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* add request context
* move initialialization to server
* use app interface instead of global app functions
* remove app context from webconn
* cleanup
* remove duplicated services
* move context to separate package
* remove finalize init method and move content to NewServer function
* restart workers and schedulers after adding license for tests
* reflect review comments
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Add support for timed DND status
- accept a date time value in api query when dnd mode for user needs to be unset
- Create a new function to handle SetDNDStatus calls
- Create a scheduled task to unset dnd mode to wahtever mode was before setting it to DND
* update schema version
* Model changes to make fields more intuitive
- move dndendtime to status model
- add new field prev status in status to keep track of previous status of user
- update db migration function
- make use of prevstatus and dndendtime from status model
* set prev status and dndendtime appropriately after unsetting dnd mode
* add json tag for dndendtime
* unset dnd status only if not changed manually by user
* update dnd statuses after server restart
* make app-layers
* fix failing tests
* don't create sched task when setting status to DND
* get only expired statuses from db
- convert end time from any timezone to utc
- store dnd end time in unix format for usability reasons
* run update dnd status only on leader
* make mocks
* fix tests
* run UpdateDNDStatusOfUsers as recurring task
* save all statuses at once in db and update UpdateDNDStatusOfUsers logic
* add app method to get timezone of user
* store dnd end time in context.Params
* set max size of prevstatus
* update status model to take endtime input as string and store in db as unix time(int64)
* Add tests for SetStatusDoNotDisturbTimed
* if dnd_end_time is not passed the call old api to set dnd mode
* fix tests
* new plugin api to use new timed dnd mode
* get and update rows in a single db query
* dnd end time will be stored in request body and not route param
* exclude statuses which has dndendtimeunix < 0
* update and get the updated dnd statuses in single db query
* add updated status to cache
* DNDEndTimeUnix and PrevStatus need not to be visible to users
* update db schema version for migration
* Keep Status and PrevStatus varchar size same
* add test to verify status is restored after dnd end time expires
* expect endtime in utc from client
- remove store method GetTimezone as no longer needed
- add documentation for SetStatusDoNotDisturbTimed
* reduce sleep time for dnd timed restore test
* more appropriate name for new api to update user status
* update db migration function
* parse and validate time before potentially triggering db query to get status of user
* add migration changes in to existing upgrade function
* not supporting un-timed dnd status via api
* don't call Srv.Store directly, call via app layer
* rename dndendtime to statuscleartime to make it suitable for custom status usage as well
* Revert "rename dndendtime to statuscleartime to make it suitable for custom status usage as well"
This reverts commit fa69152d9a3db18f1c59b34c878fb7ce494440b5.
* mysql doesn't support RETURNING clause so add tx to get and update statuses
* add UpdateDNDStatusOfUsers mock in tests
* update store mock import path
* add mock in storelib
* Add status mocks to empty store
* Close the task during server shutdown
* Do not cancel a nil task
* update squirrel queries
* remove untimed dnd test
* start recurring task to unset statuses on leadership change
* set dndTask to nil after cancelling it upon server shutdown
* new recurring task which starts at nearest rounded time of the interval
* mock Get() call for status
* return updated statuses in case of mysql
* remove unneccessary code
* add Get() mock to empty store
* fix mocking for once and all
* address review comments
fix mysql updateStatus fn
protect dndTask with mutex
minor refactors
* move runDNDStatusExpireJob to server.go and pass App as arg instead of method receiver
* frontend will send endtime in unix epoch format so get rid of double representation
* scan for all fields and not just two
* add some tests and fix review comments
* remove extra sql query and create needed result in go
* add storetest for UpdateExpiredDNDStatuses
* add migrations to latest version
* update min supported version
* add comment to fix a bug in future
* update test to expect 1 status in return
* rename UpdateUserStatusWithDNDTimeout to SetUserStatusTimedDND
* rename DNDEndTimeUnix to DNDEndTime
* cast int to int64 for equality
* fix tests and error handling
* move updating values to retrieved statuses fields outside sql transaction
* move migrations to 5.36
Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* add includeRemovedMembers flag
* fix API call in client4.go
* remove check for 'since'
* add comments
* run make app-layers
* re-run CI tests
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Removing FilesSearch feature flag
* Fixing tests
* Adding an improvement on plain text extraction
* Adding tests for plain text extraction
* Removed unneeded conversion
* Adding missed license
* Remove the feature flag from the migration
* Fixing some tests
* Updating i18n/en.json file
* MM-33601: Invalidate team member cache when updating a team scheme.
* MM-33601: Tests cache invalidation.
* MM-33601: Test fix.
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-32950: Reliable WebSockets: Basic single server
This PR adds reliable websocket support for a single server.
Below is a brief overview of the three states of a connection:
Normal:
- All messages are routed via web hub.
- Each web conn has a send queue to which it gets pushed.
- A message gets pulled from the queue, and before it
gets written to the wire, it is added to the dead queue.
Disconnect:
- Hub Unregister gets called, where the connection is just
marked as inactive. And new messages keep getting pushed
to the send queue.
If it gets full, the channel is closed and the conn gets removed
from conn index.
Reconnect:
- We query the hub for the connection ID, and get back the
queues.
- We construct a WebConn reusing the old queues, or a fresh one
depending on whether the connection ID was found or not.
- Now there is a tricky bit here which needs to be carefully processed.
On register, we would always send the hello message in the send queue.
But we cannot do that now because the send queue might already have messages.
Therefore, we don't send the hello message from web hub, if we reuse a connection.
Instead, we move that logic to the web conn write pump. We check if
the sequence number is in dead queue, and if it is, then we drain
the dead queue, and start consuming from the active queue.
No hello message is sent here.
But if the message does not exist in the dead queue, and the sequence number
is actually something that should have existed, then we set
a new connction id and clear the dead queue, and send a hello message.
The client, on receiving a new connection id will automatically
set its sequence number to 0, and make the sync API calls to manage
any lost data.
https://mattermost.atlassian.net/browse/MM-32590
```release-note
NONE
```
* gofmt
* Add EnableReliableWebSockets to the client config
* Refactoring isInDeadQueue
* Passing index to drainDeadQueue
* refactoring webconn
* fix pointer
* review comments
* simplify hasMsgLoss
* safety comment
* fix test
* Trigger CI
* Trigger CI
Co-authored-by: Devin Binnie <devin.binnie@mattermost.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* [MM-35039] - Send trial ended email
* Generations
* Use First name with fallback to username
* Use First name with fallback to username for trial ending email
* Changed the request type to POST for the remove recent custom status API
Fixed the custom status clear slash command
* Added Delete method in the remove recent custom status endpoint
* Added one new endpoint to remove recent custom status with POST method
* Added comments for the recent custom status API
Co-authored-by: Manoj <manoj@brightscout.com>
Co-authored-by: Chetanya Kandhari <availchet@gmail.com>
Co-authored-by: Manoj <77336594+manojosh@users.noreply.github.com>
Support for handling username collisions between remote clusters. Users belonging to remote clusters have their username changed to include the remote name e.g. wiggin becomes wiggin:mattermost.
@mentions are also modified so the munged username is replaced with the original username when the post is sync'd with the remote the user belongs to.
When adding remote users:
- append the remote name to the username with colon separator
- append the remote name to the email address with colon separator
- store the original username and email address in user props
- when resolving @mentions replace with the stored original username
* always add FeatureFlags to config
* update const to TitleCase
* gofmt
* update method to use 'access:all'
* update method to use 'access:all'
* fix lint
* gofmt
* added comments
* update name of access tag for any
* fix bad save
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-34002: Improve AddUserToChannel
When we would add a user to a channel, we would
check whether the user is removed from that team or not.
During LDAP sync, this check is not required because the
team member would have just been created. Hence, we
pass a boolean flag to bypass the check.
And with that done, we can freely query the replica.
https://mattermost.atlassian.net/browse/MM-34002
```release-note
NONE
```
* Refactor code
* Rename a struct field
* fix double negative
During LDAP sync, we would call AddTeamMember which had a read-after-write issue
where we would create a team member but then immediately after that
query the team member.
The same pattern was found in:
AddTeamMember
AddTeamMembers
AddTeamMemberByToken
To fix this, we just return the inserted team member from AddUserToTeam and use that
instead of query GetTeamMember again.
```release-note
NONE
```
https://mattermost.atlassian.net/browse/MM-33913
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Remote Cluster Service
- provides ability for multiple Mattermost cluster instances to create a trusted connection with each other and exchange messages
- trusted connections are managed via slash commands (for now)
- facilitates features requiring inter-cluster communication, such as Shared Channels
Shared Channels Service
- provides ability to shared channels between one or more Mattermost cluster instances (using trusted connection)
- sharing/unsharing of channels is managed via slash commands (for now)
* MM-34389: Reliable websockets: First commit
This is the first commit which makes some basic changes
to get it ready for the actual implementation.
Changes include:
- A config field to conditionally enable it.
- Refactoring the WriteMessage along with setting the deadline
to a separate method.
The basic idea is that the client sends the connection_id
and sequence_number either during the handshake (via query params),
or during challenge_auth (via added parameters in the map).
If the conn_id is empty, then we create a new one and set it.
Otherwise, we get the queues from a connection manager (TBD)
and attach them to WebConn.
```release-note
NONE
```
https://mattermost.atlassian.net/browse/MM-34389
* Incorporate review comments
* Trigger CI
* removing telemetry
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-34000: Use non-epoll mode for TLS connections
A *crypto/tls.Conn does not expose the underlying TCP connection
or even a File method to get the underlying file descriptor
like the way a *net/TCPConn does. Therefore the netpoll code would
fail to get the file descriptor.
Relevant issue here: https://github.com/mailru/easygo/issues/3
It is indeed possible to use reflect black magic to get the unexported
member, but I have found unexpected errors during writing to the websocket
by getting the file descriptor this way. I do not want to spend time investigating
this especially since this is already released.
Once this is out, we can decide on the right way to fix this, most probably
by proposing to expose the File method or some other way.
https://mattermost.atlassian.net/browse/MM-34000
```release-note
Fix an issue where websockets wouldn't work with TLS connections.
In that case, we just fall back to the way it works for Windows machines,
which is to use a separate goroutine for reader connection.
```
* Ignore logging errors on non-epoll
On non-epoll systems, we needed to return an error
to break from the loop. But in that case, there is no
need to log the error
Our proxy configuration was historically incorrect, due to which
a lot of customers have that in their setups. As a result, strictly
following the websocket RFC results in a breaking change.
For now, we transparently upgrade the version header to 1.1, if we detect 1.0.
If a client was sending 1.0, it wouldn't have worked anyways because persistent
connections were introduced from 1.1 onwards.
https://mattermost.atlassian.net/browse/MM-33836
```release-note
WebSocket handshakes done with HTTP version lower than 1.1 will result in a warning,
and the server will transparently upgrade the version to 1.1 to comply with the
websocket RFC.
This is done to work around incorrect nginx (and other proxy) configs that do not set
the proxy_http_version directive to 1.1.
This facility will be removed in a future Mattermost version and it is strongly recommended
to fix the proxy configuration to correctly use the websocket protocol.
```
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
