* MM-31717: Remove HTTP clustering
https://mattermost.atlassian.net/browse/MM-31717
```release-note
Gossip clustering mode is now GA and is no longer
available as a choice now. All cluster traffic will always
use the gossip protocol.
The config setting UseExperimentalGossip has no effect
and has only been kept for compatibility purposes.
The setting to use gossip has been removed from admin console.
Upgrade note:
If an existing system is not using gossip, then one cannot
upgrade one node in a cluster and keep the other one to the older
version, if an HA upgrade is intended. All nodes
in the cluster must use a single protocol.
Either the customer must use gossip if they want to do an HA
upgrade. Or, shut down all nodes and bring up all of them
after doing an upgrade.
```
* Trigger CI
* update translations
- ensure no logging is attempted after unit test is completed by explicitly shutting down the mock server, and ensuring no reference to testing.T is held.
Fixes a bug and adds a feature for shared channels:
- The Bug: when creating new shared channels, users that had already been sync'd via another channel were not added to the new channel's member list, since the users were not sync'd again. This PR sync's users per channel.
- The Feature: support custom statuses
Support for handling username collisions between remote clusters. Users belonging to remote clusters have their username changed to include the remote name e.g. wiggin becomes wiggin:mattermost.
@mentions are also modified so the munged username is replaced with the original username when the post is sync'd with the remote the user belongs to.
When adding remote users:
- append the remote name to the username with colon separator
- append the remote name to the email address with colon separator
- store the original username and email address in user props
- when resolving @mentions replace with the stored original username
* Use a custom user-agent when previewing links
Many websites block requests made with Go's default user-agent. We had
previous special-cased Twitter links to use a nonstandard user-agent.
This makes that behavior apply everywhere and also customizes the
user-agent to belong specifically to Mattermost.
* Correctly use custom transport for link previews
This allows us to use the custom user-agent defined in
services/httpservice/client.go.
* Stop leaking server version in custom user-agent
Since the custom user-agent is now used when previewing links, exposing
the server version could provide a vector for a malicious actor to
gather information about private deployments. To avoid this, we switch
to a generic string.
* Remove extraneous Transport creation
MakeClient already creates a transport for us, so this is unnecessary.
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Add extract documents content command
* Adding the extraction command and making the pure go pdf library as secondary option
* Improving the memory usage and docextractor interface
* Enable content extraction by default in all the instances
* Tiny improvement on archive indexing
* Adding App interface generation and the opentracing layer
* Fixing linter errors
* Addressing PR review comments
* Addressing PR review comments
* MM-34002: Improve AddUserToChannel
When we would add a user to a channel, we would
check whether the user is removed from that team or not.
During LDAP sync, this check is not required because the
team member would have just been created. Hence, we
pass a boolean flag to bypass the check.
And with that done, we can freely query the replica.
https://mattermost.atlassian.net/browse/MM-34002
```release-note
NONE
```
* Refactor code
* Rename a struct field
* fix double negative
During LDAP sync, we would call AddTeamMember which had a read-after-write issue
where we would create a team member but then immediately after that
query the team member.
The same pattern was found in:
AddTeamMember
AddTeamMembers
AddTeamMemberByToken
To fix this, we just return the inserted team member from AddUserToTeam and use that
instead of query GetTeamMember again.
```release-note
NONE
```
https://mattermost.atlassian.net/browse/MM-33913
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Remote Cluster Service
- provides ability for multiple Mattermost cluster instances to create a trusted connection with each other and exchange messages
- trusted connections are managed via slash commands (for now)
- facilitates features requiring inter-cluster communication, such as Shared Channels
Shared Channels Service
- provides ability to shared channels between one or more Mattermost cluster instances (using trusted connection)
- sharing/unsharing of channels is managed via slash commands (for now)
* MM-31094: Adds tooling to develop and test using a MySQL instance with replication lag. Adds some lazy lookups to fallback to master if results are not found.
* MM-31094: Removes mysql-read-replica from default docker services.
* MM-31094: Switches (store..SessionStore).Get and (store.TeamStore).GetMember to using context.Context.
* MM-31094: Updates (store.UsersStore).Get to use context.
* MM-31094: Updates (store.PostStore).Get to use context.
* MM-31094: Removes feature flag and config setting.
* MM-31094: Rolls back some master reads.
* MM-31094: Rolls a non-cache read.
* MM-31094: Removes feature flag from the store.
* MM-31094: Removes unused constant and struct field.
* MM-31094: Removes some old feature flag references.
* MM-31094: Fixes some tests.
* MM-31094: App layers fix.
* MM-31094: Fixes mocks.
* MM-31094: Don't reparse flag.
* MM-31094: No reparse.
* MM-31094: Removed unused FeatureFlags field.
* MM-31094: Removes unnecessary feature flags variable declarations.
* MM-31094: Fixes copy-paste error.
* MM-31094: Fixes logical error.
* MM-30194: Removes test method from store.
* Revert "MM-30194: Removes test method from store."
This reverts commit d5a6e8529bd5f4d993824c828e239d009b05e567.
* MM-31094: Conforming to make's strange syntax.
* MM-31094: Configures helper for read replica with option.
* MM-31094: Adds some missing ctx's.
* MM-31094: WIP
* MM-31094: Updates test names.
* MM-31094: WIP
* MM-31094: Removes unnecessary master reads.
* MM-31094: ID case changes out of scope.
* MM-31094: Removes unused context.
* MM-31094: Switches to a helper. Removes some var naming changes. Fixes a merge error.
* MM-31094: Removes SQLITE db driver ref.
* MM-31094: Layer generate fix.
* MM-31094: Removes unnecessary changes.
* MM-31094: Moves test method.
* MM-31094: Re-add previous fix.
* MM-31094: Removes make command for dev.
* MM-31094: Fix for login.
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Add field to config model
Config option for disabling link previews for given domains.
* Refactor functions and corresponding tests
* Expand logic for link preview
Newly added isLinkAllowedForPreview function determines whether a link
should display a preview. It gets corresponding config values consisting
of comma separated domain values, normalizes them and checks for
matches.
* Create tests for link preview restriction
* Fix formatting issue
* Add test cases where images are expected
* Apply suggestions from code review
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* Apply remaining code suggestions
* Add RestrictLinkPreviews value to telemetries
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Harrison Healey <harrisonmhealey@gmail.com>
* Adding search files api
* Fixing golangci-lint
* Adding bulk-indexing and improving a bit the name indexing for bleve and elasticsearch
* Add content extraction config migration
* Fixing a problem with document extraction
* Unapplying certain changes moved to other PR
* Fixing tests
* Making extract content app migration a private method
* Addressing PR review comments
* Addressing PR review comments
* Adding feature flag
* Removing debug string
* Fixing imports
* Fixing linting errors
* Do not migrate the config if the feature flag is not enabled
* Fix tests
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* services/bleve: only log update conf when there is an actual change
* Update services/searchengine/bleveengine/bleve.go
Co-authored-by: Miguel de la Cruz <miguel@mcrx.me>
Co-authored-by: Miguel de la Cruz <miguel@mcrx.me>
* Adding bulk-indexing and improving a bit the name indexing for bleve and elasticsearch
* Update services/searchengine/bleveengine/bleve.go
Co-authored-by: Mario de Frutos Dieguez <mario@defrutos.org>
* Update store/sqlstore/file_info_store.go
Co-authored-by: Mario de Frutos Dieguez <mario@defrutos.org>
* Update store/sqlstore/file_info_store.go
Co-authored-by: Mario de Frutos Dieguez <mario@defrutos.org>
* Adding tests requested in the PR review
* fixing tests
* Adding a feature flag to avoid indexing files before the feature is released
* Fixing i18n
Co-authored-by: Mario de Frutos Dieguez <mario@defrutos.org>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Isolate more the mfa service
* Simplifing the mfa service
* Removing channels and adding waitgroup
* Migrating mfa service to regular errors
* Fixing tests
* i18n extract
* Addressing PR review comments
* Removing unneeded struct
* Create the system console setting and send to webapp
* MI-1145: Add custom status APIs
* MI-1145 Add slash commands to set and clear status
* Add validation for custom status API
* Trim custom status message
* Code refactoring
- Run gofmt
- Rename constants
* Remove sendUserUpdated webhook event
* Fix recent custom status length
* Update error conditions
* Disable /status slash command when config setting is off
* MI-1155: Create the feature flag for custom status APIs and slash commands
* Move recent custom statuses to user preferences (#7)
* Move recent custom statuses to user preferences
* Code refactoring and feedback changes
* Update slash command text and emoji regex
* Make the custom status feature flag off by default
* Update SetCustomStatus, handle recents not set better
* Update status codes
* Update slash command handling
* Add telementry settings
* Fix i18n order
* Revert "Fix i18n order"
This reverts commit 499f7eaca8180336f5bcca360cc0133365899e08.
* Update i18n strings
* Reduce the coupling of the mailservice with the rest of the application
* Fixing tests in CI
* Simplifiying mailservice config
* Addressing PR review comments
* Fixing tests
* Removing unnecesary type definition
* Fixing ServerName usage
* Reducing the filestore dependencies from the rest of the source code
* Making more generic config conversion to FileBackendSettings
* Fixing usage of the NewFileBackend function
* Fixing more usages of the NewFileBackend function
* Fix some linter errors
* Fix more linter errors
* Fixing some unit tests
* Fixing linter problem
* Addressing PR review comments
* Simplifing the CopyFile for tests
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* init commit
* clean up the code
* make mocks
* fix translations
* mocks and lint fixes
* add tests
* little fixes
* Update i18n/en.json
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
* Update i18n/en.json
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
* Update i18n/en.json
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
* Update i18n/en.json
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
* Update i18n/en.json
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
* Address Comments
* fix i18n
* update api endpoint
* add enable file and file level for conditional show of banner
* Address Comments
* Make it more clear about returns
* Create zip file utility function
* update en.json
* address comments
* write tests
* check for data in test
* remove warning string
* Correct expected and actual
* set database through environment variables
* reset environment variable at end of test
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
