459 Коммитов

Автор SHA1 Сообщение Дата
Rahim Rahman
55c2d57f70 fix(MM-60645): missing emoji on mobile (#30001)
* fix(MM-60645): missing emoji on mobile

* removing the `:` completely from being stored or being sent to the server.

* proper formatting?
2025-02-09 21:38:29 -07:00
unified-ci-app[bot]
144986c1ad Update latest minor version to 10.6.0 (#30138)
Automatic Merge
2025-02-07 11:20:09 +02:00
Agniva De Sarker
c4718e4542 MM-60211: Bump dependencies (#30005)
https://mattermost.atlassian.net/browse/MM-60211
```release-note
NONE
```

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-02-06 21:48:08 +05:30
Julien Tant
bcc395d139 [MM-62552] Custom Profile Attributes: use json.RawMessage for the value. (#29989)
* refactor: Move property value sanitization to model layer

* feat: Add value sanitization for custom profile attributes

* refactor: Update custom profile attributes to use json.RawMessage

* refactor: Update patchCustomProfileAttribute to handle json.RawMessage directly

* refactor: Refactor custom profile attributes handler with improved validation

* refactor: Rename `patchCustomProfileAttribute` to `patchCPAValues`

* refactor: Replace ReturnJSON with json.NewEncoder and add error logging

* feat: Add encoding/json import to property_value.go

* refactor: Update property value tests to use json.RawMessage

* fix: Convert string value to json.RawMessage in property value test

* fix: Convert string literals to json.RawMessage in property value tests

* fix: Add missing encoding/json import in custom_profile_attributes.go

* fix: Preserve JSON RawMessage type in listCPAValues function

* fix: Update custom profile attributes test to use json.RawMessage

* feat: Add json import to custom_profile_attributes_test.go

* refactor: Update ListCPAValues and PatchCPAValues to use json.RawMessage

* refactor: Rename `actualValue` to `updatedValue` in custom profile attributes test

* refactor: Improve user permission and audit logging for custom profile attributes patch

* refactor: Optimize CPA field lookup by using ListCPAFields() and map

* fix: Correct user ID reference in custom profile attributes patch endpoint

* refactor: Change patchCPAValues to use map[string]json.RawMessage for results

* refactor: format and fix tests

* test: Add comprehensive unit tests for sanitizePropertyValue function

* test: Add test case for invalid property value type

* feat: Use `model.NewId()` to generate valid IDs in custom profile attributes tests

* refactor: Replace hardcoded IDs with dynamic variables in custom profile attributes test

* refactor: restore variable name

* refactor: drop undesired changes

* chore: refresh app layers

* feat: Update API definition to support string or string array values for custom profile attributes

* test: Add test cases for multiselect custom profile attribute values

* test: Add tests for multiselect custom profile attribute values

* test: Isolate array value test in separate t.Run

* test: Add test case for multiselect array values in custom profile attributes

* refactor: Move array value test from TestCreateCPAField to TestPatchCPAValue

* test: Update custom profile attributes test assertions

* test: add test case for handling array values in GetCPAValue

* test: Add array value tests for property value store

* refactor(store): no need to convert to json the rawmessage

* chore: lint

* i18n

* use model to interface with sqlx

* fix: Allow empty strings for text, date, and select profile attributes

* refactor: Filter out empty strings in multiselect and multiuser fields

* refactor: Update multiuser field sanitization to validate and error on invalid IDs

* refactor: Simplify sanitizePropertyValue function with reduced code duplication

* fix: Allow empty user ID in custom profile attribute sanitization

* refactor: Convert comment-based subtests to nested t.Run in TestSanitizePropertyValue

* refactor: Convert comment-based subtests to nested t.Run tests in TestSanitizePropertyValue

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-02-05 17:21:22 +00:00
Agniva De Sarker
ae9e6174e5 [AI assisted]: Improve system console statistics performance (#29899)
```release-note
NONE
```

Co-authored-by: Matthew Birtch <mattbirtch@gmail.com>
2025-02-04 21:54:01 +05:30
Miguel de la Cruz
9f172fed70 Update the property field model to interface with SQLX (#29987)
* Update the property field model to interface with SQLX

* Fix tests
2025-01-30 03:32:24 +00:00
Jesús Espino
f1acdce42c Removing opentracing (#29965)
* Removing opentracing

* Fixing CI
2025-01-29 07:45:13 +01:00
Doug Lauder
565c8aa42d Audit usage of the searchPosts API (#29909)
* Audit calls to searchPosts API

* Update server/public/model/post_search_results.go

Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>

* Update server/public/model/post_search_results.go

* Update server/public/model/post_search_results.go

---------

Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-01-28 22:21:05 -05:00
Christopher Poile
87ea6d8b1c [MM-62537] Fix: Integration action nil ptr dereference (#29986)
* guard against nil dereference; lock down with tests

* fix test name
2025-01-28 13:46:12 -05:00
Scott Bishel
b26c43e716 Permission changes (#29570)
* update permission mistakes

* add getAnalytics to TeamStatistics

* add PermissionGetAnalytics to ReadTeamStatistics

* add mocks for migrations

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-01-27 16:28:27 +00:00
Julien Tant
b41b968eb8 restrict access to channel converstion to public for non team admin+ (#29819) 2025-01-23 09:42:20 -07:00
Agniva De Sarker
05cdb36886 MM-49353: Setup intermediate signal handlers for plugin shutdown (#28653)
There are several steps that a server runs through inside (*Server).Start
after ch.initPlugins() till the signal handler is reached which handles
the server shutdown procedure.

The issue arises when the server is shutdown after ch.initPlugins() completes
but before (*Server).Start finishes. In that case, the plugins are all started
but they won't be shut down cleanly.

To fix this edge-case, we set up an intermediate signal handler, which
attaches itself as soon as ch.initPlugins is finished, allowing us to run
the cleanup code in case the shutdown happens before (*Server).Start finishes.

And when we do reach the main signal handler, we don't need this intermediate
handler any more. So we just reset the handlers and use the main signal handler
which takes care of shutting down the whole server.

Note: This is still not 100% bug-proof because ch.initPlugins() will initialize
_all_ plugins, and the shutdown can happen just after one plugin is initialized.
To handle that case will require the need to set up signal handlers after every
plugin init which feels like overkill to me.

A sample flow diagram to visualize better:

Edge-case
server.Start()
|
ch.initPlugins()
|
<ctrl-c>
|
execute signal handler, os.Exit(1)

Happy-path
server.Start()
|
ch.initPlugins()
|
server.Start() finished
|
reset old signal handler
|
setup main signal handler
|
server runs on as usual until shutdown

https://mattermost.atlassian.net/browse/MM-49353

```release-note
NONE
```
2025-01-21 11:15:19 +05:30
Agniva De Sarker
cb75a20c54 MM-61904: Make reliable websockets work in HA (#29489)
We do a cluster request to get the active and dead queues
from other nodes in the cluster to sync any missing
information.

We check the dead queue in the other nodes to see
if there's been any message loss or not. Accordingly,
we send just the active queue or both active and dead queues.

There's still an edge case that is left out where
a client could have potentially connected and reconnected
to multiple nodes leaving multiple active queues
in multiple nodes. We don't handle this scenario
because then potentially we need to create
a slice of sendQueueSize * number_of_nodes. And then
this can happen again, leading to an infinite increase
in sendQueueSize.

We leave this edge-case to Redis, acknowledging
a limitation in our architecture.

In this PR, when there's no message loss, we just
take the active queue from the last node it connected
to.

And if there's message loss where the client's
seqNum is within the last node's dead queue, we also
handle that.

But if there's severe message loss where the client's
seqNum falls within the dead queue of another node, then
we just send the data from that node to reconstruct the
data as much as possible. It could be possible to set
a new connection ID in this case, but this involves
more data transfer always from all nodes and recomputing
the state in the requestor node.

https://mattermost.atlassian.net/browse/MM-61904

```release-note
NONE
```

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-01-17 11:11:32 +05:30
Rahim Rahman
540408545a feat(MM-61865): Add mobile client content load network metrics (#29601)
* MM-61865: Add mobile client content load network metrics

* added new common label

* renaming from MobileClientContentLoad to MobileClientNetworkRequests
* content_load_group => network_request_group

* refactor more NetworkRequest-* changes

* replace contentLoadGroup to networkRequestGroup

* new metrics elapsedTime

* Refactor urlCount to totalRequests

* add averageSpeed metric
* replace contentLoadGroup with networkRequestGroup
* use h.Labels vs commonLabels for network_request_group

* add agent

* add effective latency metrics

* add total parallel requests & total sequential

* mocks generated by mockery

* did a bit of cleanup and sorting

* formatting

* updated the AcceptedNetworkRequestGroups

* cleanup and sorting
2025-01-16 11:03:41 -07:00
Ben Schumacher
8d4bf4bae0 [MM-54288] Support Packet V2 (#29403) 2025-01-13 20:23:09 +01:00
Miguel de la Cruz
ca34c6a03f Custom profile attributes field endpoints (#29662)
* Adds the main Property System Architecture components

This change adds the necessary migrations for the Property Groups,
Fields and Values tables to be created, the store layer and a Property
Service that can be used from the app layer.

* Adds Custom Profile Attributes endpoints and app layer

* implement get and patch cpa values

* run i18n-extract

* Update property field type to use user instead of person

* Update PropertyFields to allow for unique nondeleted fields and remove redundant indexes

* Update PropertyValues to allow for unique nondeleted fields and remove redundant indexes

* Use StringMap instead of the map[string]any on property fields

* Add i18n strings

* Revert "Use StringMap instead of the map[string]any on property fields"

This reverts commit e2735ab0f8589d2524d636419ca0cb144575c4d6.

* Cast JSON binary data to string and add todo note for StringMap use

* Add mocks to the retrylayer tests

* Cast JSON binary data to string in property value store

* Check for binary parameter instead of casting to string for JSON data

* Fix bad merge

* Check property field type is one of the allowed ones

* Avoid reusing err variable to be explicit about the returned value

* Merge Property System Migrations into one file

* Adds NOT NULL to timestamps at the DB level

* Update stores to use tableSelectQuery instead of a slice var

* Update PropertyField model translations to be more explicit and avoid repetition

* Update PropertyValue model translations to be more explicit and avoid repetition

* Use ExecBuilder instead of ToSql&Exec

* Update property field errors to add context

* Ensure PerPage is greater than zero

* Update store errors to give more context

* Use ExecBuilder in the property stores where possible

* Add an on conflict suffix to the group register to avoid race conditions

* Remove user profile API documentation changes

* Update patchCPAValues endpoint and docs to return the updated information

* Merge two similar error conditions

* Use a route function for ListCPAValues

* Remove badly used translation string

* Remove unused get in register group method

* Adds input sanitization and validation to the CPA API endpoints

* Takes login outside of one test case to make it clear it affects multiple t.Runs

* Fix wrap error and return code when property field has been deleted

* Fix receiver name

* Adds comment to move the CPA group ID to the db cache

* Set the PerPage of CPA fields to the fields limit

* Update server/channels/app/custom_profile_attributes_test.go

Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com>

* Standardize group ID access

* Avoid polluting the state between tests

* Use specific errors for the retrieval of CPA group

---------

Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com>
2025-01-13 18:12:38 +01:00
Harshil Sharma
6e5a67caec Feature edit attachments (#29769)
* Updated patch/update post API to allow file modification (#29447)

* WIP

* WIP

* Atatched new files ton post

* WIP: deleting removed files

* Deleted removed files and invalidated file metadata cache

* removed file ignore logif from update post API

* Added TestFindExclusives

* Added tests for DeleteForPostByIds

* Added app layer tests

* Added tests

* Added API level tests

* test enhancements

* Fixed a test

* Edit history include file metadata (#29505)

* Send file metadata in edit history metadata

* Added app tests

* Added store tests

* Added tests for populateEditHistoryFileMetadata{

* Added cache to avoid repetitigve DB calls for edits with only message changes

* Added API tests

* i18m fix

* removed commented code

* Improved test helper

* Show attachments in edit history RHS (#29519)

* Send file metadata in edit history metadata

* Added app tests

* Added store tests

* Added tests for populateEditHistoryFileMetadata{

* Added cache to avoid repetitigve DB calls for edits with only message changes

* Added API tests

* i18m fix

* WIUP: displa files in edit

* removed commented code

* Displayed file in edit history

* Handled file icon

* Fixed closing history component on clicking on file

* Simplified selector

* Simplified selector

* Improved test helper

* Disabled action menu on edit history file

* Added tests

* Improved selector

* Updated snapshot

* review Fixes

* restructured componnets

* Updated test

* Updated test

* Restore post api (#29643)

* Restore post version API WIP

* Undelete files WIP

* Added store tests

* Created post restore API

* Updated updatepost safeUpdate signature

* review fixex and improvements

* Fixed an app test

* Added API laer tests

* Added API tests and OpenAPI specs

* Fixed a typo

* Allow editing files when editing posts (#29709)

* WIP - basic view files when editing post

* Cleanup

* bg color

* Added text editor tests for files

* WIP

* WIP

* removed debug log

* Allowed admin to add and remove files on someone else's post

* Handled drafts and scheduled posts

* linter fixes

* Updated snapshot

* server test fix

* CI

* Added doc

* Restore post api integration (#29719)

* WIP - basic view files when editing post

* Cleanup

* bg color

* Added text editor tests for files

* WIP

* WIP

* removed debug log

* Allowed admin to add and remove files on someone else's post

* Handled drafts and scheduled posts

* linter fixes

* Updated snapshot

* server test fix

* Used new API to restore post

* handled edut limit and undo

* lint fix

* added comments

* Fixed edit post item tests

* Fixed buttons

* Aded snapshots

* fix test

* Updated snapshot

* Minor fixes

* fixed snapshot

* Edit file dnd area (#29763)

* dnd wip

* DND continued

* Supported multiple unbind dragster funcs

* lint fixes

* Got center channel file drop working when editing a post

* file dnd working with center channel and rhs

* file dnd working with center channel and rhs

* removed unneeded stopPropogation calls

* cleanup

* DND overlay fix

* Lint fix

* Advanced text editor test updates for file upload overlay

* fixed use upload hook tests

* Updated some more snapshots

* minor cleanup

* Updated i18n

* removed need of array for dragster unbind events

* lint fixes

* edit history cursor

* Fixed bugu causing faliure to delete empty posts (#29778)

* Files in restore confirmation (#29781)

* Added files to restore post confirmation dialog

* Fixed post restore toast colors

* Fixed restore bug

* Fixed restore confirmation toast tests

* a11y improvement and modal width fix

* Edit attachment misc fixes (#29808)

* Removed single image actions in restore post confirmation dialog

* Fixed file drop overlay size and position

* Made edit indiator accessible

* Lint fix

* Added bunch of more tests

* ANother test migrated from enzyme to react testing library

* More test enhancements

* More test enhancements

* More test enhancements

* lint fixes

* Fixed  a test

* Added missing snapshots

* Test fixes
2025-01-13 18:16:56 +05:30
Miguel de la Cruz
ecdce71fc4 Adds the main Property System Architecture components (#29644)
* Adds the main Property System Architecture components

This change adds the necessary migrations for the Property Groups,
Fields and Values tables to be created, the store layer and a Property
Service that can be used from the app layer.

* Update property field type to use user instead of person

* Update PropertyFields to allow for unique nondeleted fields and remove redundant indexes

* Update PropertyValues to allow for unique nondeleted fields and remove redundant indexes

* Use StringMap instead of the map[string]any on property fields

* Add i18n strings

* Revert "Use StringMap instead of the map[string]any on property fields"

This reverts commit e2735ab0f8589d2524d636419ca0cb144575c4d6.

* Cast JSON binary data to string and add todo note for StringMap use

* Add mocks to the retrylayer tests

* Cast JSON binary data to string in property value store

* Check for binary parameter instead of casting to string for JSON data

* Check property field type is one of the allowed ones

* Avoid reusing err variable to be explicit about the returned value

* Merge Property System Migrations into one file

* Adds NOT NULL to timestamps at the DB level

* Update stores to use tableSelectQuery instead of a slice var

* Update PropertyField model translations to be more explicit and avoid repetition

* Update PropertyValue model translations to be more explicit and avoid repetition

* Use ExecBuilder instead of ToSql&Exec

* Update property field errors to add context

* Ensure PerPage is greater than zero

* Update store errors to give more context

* Use ExecBuilder in the property stores where possible

* Add an on conflict suffix to the group register to avoid race conditions

* Remove badly used translation string

* Remove unused get in register group method

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-01-13 11:41:44 +00:00
Claudio Costa
086e6d9df6 [MM-62066] Allow using plugin settings defined in manifest as fallback when using custom sections and plugin is disabled (#29500)
* Allow using plugin settings defined in manifest as fallback when using custom sections and plugin is disabled

* Update server/public/model/manifest.go

Co-authored-by: Christopher Poile <cpoile@gmail.com>

* Add custom setting case

---------

Co-authored-by: Christopher Poile <cpoile@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-01-13 11:19:59 +01:00
enzowritescode
76eee25206 Fix IDE warninings about comment (#29818) 2025-01-10 17:33:12 -07:00
Christopher Poile
aba4434dab MM-59966 - Compliance Export overhaul - feature branch (#29789)
* [MM-59089] Add a compliance export constant (#27919)

* add a useful constant

* i18n

* another constant

* another i18n

* [MM-60422] Add GetChannelsWithActivityDuring (#28301)

* modify GetUsersInChannelDuring to accept a slice of channelIds

* add GetChannelsWithActivityDuring

* add compliance export progress message; remove unused custom status

* linting

* tests running too fast

* add batch size config settings

* add store tests

* linting

* empty commit

* i18n changes

* fix i18n ordering

* MM-60570 - Server-side changes consolidating the export CLI with server/ent code (#28640)

* add an i18n field; add the CLI's export directory

* int64 -> int

* Add UntilUpdateAt for MessageExport and AnalyticsPostCount

to merge

* remove now-unused i18n strings

* add TranslationsPreInitFromBuffer to allow CLI to use i18n

* use GetBuilder to simplify; rename TranslationsPreInitFromFileBytes

* [MM-59089] Improve compliance export timings (#1733 - Enterprise repo)

* MM-60422 - Performance and logic fixes for Compliance Exports (#1757 - Enterprise repo)

* MM-60570 - Enterprise-side changes consolidating the export CLI with server/ent code (#1769 - Enterprise repo)

* merge conflicts; missed file from ent branch

* MM-61038 - Add an option to sqlstore.New (#28702)

remove useless comment

add test

add an option to sqlstore.New

* MM-60976: Remove RunExport command from Mattermost binary (#28805)

* remove RunExport command from mattermost binary

* remove the code it was calling

* fix i18n

* remove test (was only testing license, not functionality)

* empty commit

* fix flaky GetChannelsWithActivityDuring test

* MM-60063: Dedicated Export Filestore fix, redo of #1772 (enterprise) (#28803)

* redo filestore fix #1772 (enterprise repo) on top of MM-59966 feature

* add new e2e tests for export filestore

* golint

* ok, note to self: shadowing bad, actually (when there's a defer)

* empty commit

* MM-61137 - Message export: Support 7.8.11 era dbs (#28824)

* support 7.8.11 era dbs by wrapping the store using only what we need

* fix flaky GetChannelsWithActivityDuring test

* add a comment

* only need to define the MEFileInfoStore (the one that'll be overridden)

* blank commit

* MM-60974 - Message Export: Add performance metrics (#28836)

* support 7.8.11 era dbs by wrapping the store using only what we need

* fix flaky GetChannelsWithActivityDuring test

* add a comment

* only need to define the MEFileInfoStore (the one that'll be overridden)

* performance metrics

* cleanup unneeded named returns

* blank commit

* MM-60975 - Message export: Add startTime and endTime to export folder name (#28840)

* support 7.8.11 era dbs by wrapping the store using only what we need

* fix flaky GetChannelsWithActivityDuring test

* add a comment

* only need to define the MEFileInfoStore (the one that'll be overridden)

* performance metrics

* output startTime and endTime in export folder

* empty commit

* merge conflict

* MM-60978 - Message export: Improve xml fields; fix delete semantics (#28873)

* support 7.8.11 era dbs by wrapping the store using only what we need

* fix flaky GetChannelsWithActivityDuring test

* add a comment

* only need to define the MEFileInfoStore (the one that'll be overridden)

* performance metrics

* output startTime and endTime in export folder

* empty commit

* add xml fields, omit when empty, tests

* fix delete semantics; test (and test for update semantics)

* clarify comments

* simplify edited post detection, now there's no edge case.

* add some spacing to help fast running tests

* merge conflicts/updates needed for new deleted post semantics

* linting; fixing tests from upstream merge

* use SafeDereference

* linting

* stronger typing; better wrapped errors; better formatting

* blank commit

* goimports formatting

* fix merge mistake

* minor fixes due to changes in master

* MM-61755 - Simplifying and Support reporting to the db from the CLI (#29281)

* finally clean up JobData struct and stringMap; prep for CLI using db

* and now simplify using StringMapToJobDataWithZeroValues

* remove unused fn

* create JobDataExported; clean up errors

* MM-60176 - Message Export: Global relay cleanup (#29168)

* move global relay logic into global_relay_export

* blank commit

* blank commit

* improve errors

* MM-60693 - Refactor CSV to use same codepath as Actiance (#29191)

* move global relay logic into global_relay_export

* blank commit

* refactor (and simplify) ExportParams into shared

* blank commit

* remove unused fn

* csv now uses pre-calculated joins/leaves like actiance

* improve errors

* remove nil post check; remove ignoredPosts metric

* remove unneeded copy

* MM-61696 - Refactor GlobalRelay to use same codepath as Actiance (#29225)

* move global relay logic into global_relay_export

* blank commit

* refactor (and simplify) ExportParams into shared

* blank commit

* remove unused fn

* csv now uses pre-calculated joins/leaves like actiance

* remove newly unneeded function and its test. goodbye.

* refactor GetPostAttachments for csv + global relay to share

* refactor global_relay_export and fix tests (no changes to output)

* improve errors

* remove nil post check; remove ignoredPosts metric

* remove unneeded copy

* remove unneeded nil check

* PR comments

* MM-61715 - Generalize e2e to all export types 🤖  (#29369)

* move global relay logic into global_relay_export

* blank commit

* refactor (and simplify) ExportParams into shared

* blank commit

* remove unused fn

* csv now uses pre-calculated joins/leaves like actiance

* remove newly unneeded function and its test. goodbye.

* refactor GetPostAttachments for csv + global relay to share

* refactor global_relay_export and fix tests (no changes to output)

* improve errors

* remove nil post check; remove ignoredPosts metric

* remove unneeded copy

* remove unneeded nil check

* PR comments

* refactor isDeletedMsg for all export types

* fix start and endtime, nasty csv createAt bug; bring closer to Actiance

* align unit tests with new logic (e.g. starttime / endtime)

* refactor a TimestampConvert fn for code + tests

* bug: pass templates to global relay (hurray for e2e tests, otherwise...)

* add global relay zip to allowed list (only for tests)

* test helpers

* new templates for e2e tests

* e2e tests... phew.

* linting

* merge conflicts

* unexport PostToRow; add test helper marker

* cleanup, shortening, thanks to PR comments

* MM-61972 - Generalize export data path - Actiance (#29399)

* extract and generalize the export data generation functions

* finish moving test (bc of previous extraction)

* lift a function from common -> shared (to break an import cycle)

* actiance now takes general export data, processes it into actiance data

* bring tests in line with correct sorting rules (upadateAt, messageId)

* fixups, PR comments

* turn strings.Repeat into a more descriptive const

amended: one letter fix; bad rebase

* MM-62009 - e2e clock heisenbug (#29434)

* consolidate assertions; output debuggable diffs (keeping for future)

* refactor test output generator to generators file

* waitUntilZeroPosts + pass through until to job = fix all clock issues

* simplify messages to model.NewId(); remove unneeded waitUntilZeroPosts

* model.NewId() -> storetest.NewTestID()

* MM-61980 - Generalize export data path - CSV (#29482)

* simple refactoring

* increase sleep times for (very) rare test failures

* add extra information to the generic export for CSV

* adj Actiance to handle new generic export (no difference in its output)

* no longer need mergePosts (yay), move getJoinLeavePosts for everyone

* adjust tests for new csv semantics (detailed in summary)

* and need to add the new exported data to the export_data_tests

* rearrange csv writing to happen after data export (more logical)

* linting

* remove debug statements

* figured out what was wrong with global relay e2e test 3; solid now

* PR comments

* MM-61718 - Generalize export data path - Global Relay (#29508)

* move global relay over to using the generalized export data

* performance pass -- not much can be done

* Update server/enterprise/message_export/global_relay_export/global_relay_export.go

Co-authored-by: Claudio Costa <cstcld91@gmail.com>

---------

Co-authored-by: Claudio Costa <cstcld91@gmail.com>

* MM-62058 - Align CSV with Actiance (#29551)

* refactoring actiance files and var names for clarity

* bug found in exported attachments (we used to miss some start/ends)

* changes needed for actiance due to new generic exports

* bringing CSV up to actiance standards

* fixing global relay b/c of new semantics (adding a note on an edge case)

* aligning e2e tests, adding comments to clarify what is expected/tested

* necessary changes; 1 more test for added functionality (ignoreDeleted)

* comment style

* MM-62059 - Align Global Relay with Actiance/CSV; many fixes (#29665)

* core logic changes to general export_data and the specific export paths

* unit tests and e2e tests, covering all new edge cases and all logic

* linting

* better var naming, const value, and cleaning up functions calls

* MM-62436 - Temporarily skip cypress tests that require download link (#29772)

---------

Co-authored-by: Claudio Costa <cstcld91@gmail.com>
2025-01-10 16:56:02 -05:00
Ben Schumacher
ca439e73f3 Don't use fmt.Sprintf for constant strings (#29799) 2025-01-10 10:47:20 +01:00
Nadav Tasher
baf9bf82c9 Bugfix - Enforce URL length check in LinkMetadata before attempting to insert to database (#29589)
* Add missing URL length validation in LinkMetadata

* Added MaxURLLength constant

* Add test to validate new URL length limit

* Add comment about table constraint

* Use a seperate error string for URL length errors

* Added error string fpr URL length

* Updated error message

* Update en.json

* Fixed gofmt error in link_metadata.go

* use right txn id

```release-note
NONE
```

---------

Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-01-08 20:02:39 +05:30
unified-ci-app[bot]
22856b4d64 Update latest minor version to 10.5.0 (#29779)
Automatic Merge
2025-01-08 10:48:31 +02:00
Agniva De Sarker
647c9b6808 [MM-62293]: Remove Elasticsearch from translation strings (#29728)
We cannot simply use Elasticsearch any more because
we support Opensearch as well.

Therefore changing Elasticsearch to just say Search.

Note: There are still some cases where Elasticsearch
is used. That is intentional and necessary.

```release-note
NONE
```
2025-01-08 11:11:59 +05:30
Thomas Decaux
b61580f87f fix(pluginapi/store): init master db if no replica (#29674)
* fix(pluginapi/store): init master db if no replica

GetReplicaDB return nil because masterDB is not initialized.

* chore: add test
2025-01-02 13:51:22 +05:30
Ibrahim Serdar Acikgoz
5e7e441623 model/config: filtering should work with structs too (#29663) 2024-12-20 19:29:02 +01:00
Claudio Costa
8c41ec75db httpservice: improve validation of proxied URLs (#29600) 2024-12-19 17:55:42 +00:00
Ibrahim Serdar Acikgoz
e76786278b [MM-60083] export: do not export DMs/GMs if users are deleted (#28854) 2024-12-19 16:47:52 +01:00
Ibrahim Serdar Acikgoz
de59770797 [MM-51468] import: allow usernames to be uppercase (#29530) 2024-12-17 13:27:53 +00:00
Ibrahim Serdar Acikgoz
424ce2b8db [MM-59503] export: enable exporting configuration with mmctl (#28412) 2024-12-17 10:23:52 +01:00
fume4mattermost
4a5994906c [MM-56031] Add DeleteSidebarCategory method and channel category API tests (#29161)
* Add DeleteSidebarCategory method and channel category API tests

* Add test to verify channels move to default categories when custom category is deleted

* rename channel category tests and remove duplicate context

* TestNotifications fix

---------

Co-authored-by: Fume <contact@fumedev.com>
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2024-12-16 16:29:43 -05:00
Pavel Mokeev
8b86e1276e GH-27059: Rewrite error messages about msg length (#29252)
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
2024-12-10 14:32:13 +01:00
Agniva De Sarker
e6c0ed4a1b MM-62085: Handle nil post priority (#29496)
https://mattermost.atlassian.net/browse/MM-62085

```release-note
NONE
```
2024-12-06 10:10:41 +05:30
unified-ci-app[bot]
421001c981 Update latest minor version to 10.4.0 (#29474)
Automatic Merge
2024-12-04 19:47:36 +01:00
Nick Misasi
027ac51b81 Remove write_restrictable from AdvancedLoggingJSON (#29424) 2024-12-04 13:29:03 -05:00
Agniva De Sarker
4ec4b4d525 MM-61886: Add actionable page navigation metrics (#29332)
Page load is one of the metrics that we track and present
to MLT. However, in its current form, it is not very
actionable because it also contains the network latency.

We split the whole metric into these parts:
startTime
|
responseStart = TTFB
|
responseEnd = TTLB
|
domInteractive = Start of processing phase
|
loadEventEnd = Load complete

This gives us better visibility into exactly
which phase in the load process is slow.

I have experimented with other metrics like
- domContentLoadedEventStart
- domContentLoadedEventEnd
- domComplete

and observed that they do not have sufficient
gaps in the timespan to have any relevance.

Additionally, I have moved TTFB from being a
web vitals metric to being tracked from the performance
metrics to remain consistent with the other navigation
metrics measured.

Lastly, I took this chance to improve some of the
validation errors that we threw to include more
context into the input that was passed and why
does it fail.

This also meant that I had to change the tests
to check for error strings rather than direct
errors which is a bad thing, but I don't think
it's worth the effort trying to have named error
variables for all of them.

https://mattermost.atlassian.net/browse/MM-61886

```release-note
NONE
```
2024-11-29 11:24:35 +05:30
Julien Tant
3b1eb64e02 [MM-51201/MM-60406/MM-60404] CrossTeam Search posts and files (#28478)
* poc - wip

* add search files across teams

* eslint

* fix existing tests

* fix webapp style

* fix test

* add api doc

* change initial state in test

* add tests on API

* add tests on file info layer

* fix file search tags

* add rhs reducer test

* reset team selected when the RHS is suppressed

* change css to reflect UI

* fix style

* fix doc wording

* make getSearchTeam return currentTeamId when value is not set

* await is unnecessary

* revert boolean check and add test

* add comment to getSearchTeam to let dev knows it defaults to currentTeam

* remove redundant team check

* simplfy test

* fix style check

---------

Co-authored-by: Caleb Roseland <caleb@calebroseland.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2024-11-21 13:40:46 -07:00
Pattara Kiatisevi
f0280d6dd4 Thai tone marks and vowels at the end of the search term is wrongly removed. (#29071)
Co-authored-by: sukoom pornsuksiri <sukoom@mm.co.th>
2024-11-21 13:47:48 +00:00
Claudio Costa
3af3af0b25 [MM-60792] Add support for loading experimental (e.g., in progress) languages (#29009)
* Add support for loading experimental (e.g., in progress) languages

* Fix e2e config

* Improve language labels

* Improvements
2024-11-14 15:02:00 -06:00
Elias Nahum
701d1dbd68 optionally exclude threads in direct messages (#29042)
* optionally exclude threads in direct messages

* add excludeDirect option in client4

* Skip flaky test

* refactor double negate

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-11-14 16:57:15 +08:00
Agniva De Sarker
63c959f1d3 Fail fast while connecting to replicas (#29212)
Since we have the auto-retry mechanism for replicas,
there is not much benefit in trying 5 times before deciding
to move on.

In the earlier model, we would fail the server startup,
so it made sense to check as many times as possible.

Also reducing the sleep interval so that we can improve
the boot up time in case of a bad replica.
```release-note
NONE
```

* fix: Add job name to the publish report step

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Antonis Stamatiou <stamatiou.antonis@gmail.com>
2024-11-14 09:45:52 +05:30
Ben Schumacher
053d0b5f0a [MM-61140] Allow plugins to add Support Packet data without UI elements (#28833)
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
2024-11-13 11:20:39 +01:00
Nadav Tasher
5e47c97db4 Added support for S3 storage classes (#28319)
* Added support for S3 storage classes

* Added missing translations for S3 storage class

* Changed default storage class values to preserve original behaviour

* Changed storage class description and example

* Fix translations ordering

* Change configuration defaults to empty strings

* Remove redundant empty string check

* Validate storage class variable against constants from S3 API docs

* Validate export storage class against constants

* Use slices for config validation

* Applied patch for translation ordering
2024-11-13 12:29:44 +05:30
Daniel Espino García
118d0346ee Add test notification tool (#28334)
* Add test notification tool

* Add frontend styles

* Remove option from admin view

* Refactor create post and add translations

* Fix several CI errors

* Fix API and frontend snapshots

* Refactor trailing and leading icon on buttons

* Add different button states

* i18n-extract

* Fix wrong text

* Add tests

* Fix wrong string

* Fix test

* feat: E2E send test notifications (#28371)

* Refactor send desktop notification

* Address rest of the feedback

* Fix tests

* Add correct link

* Fix test

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: yasserfaraazkhan <attitude3cena.yf@gmail.com>
2024-11-08 13:57:06 +01:00
M-ZubairAhmed
30a6ddc995 [MM-61439] New Users: Change default setting for mobile push notifications to 'online, away, or offline'(#29142) 2024-11-08 12:21:27 +00:00
Nicolas Le Cam
c90e562528 Migrate mockery to packages feature (#29013) 2024-11-07 12:48:11 +01:00
Pablo Vélez
fbee99e870 MM-60124-emit and read create scheduled post WS event (#28844)
* MM-60124-emit and read create scheduled post WS event

* fix unit test; reorder get team id logic; use userid for ws events

* MM-60145 - ws events on update scheduled posts (#28871)

* MM-60145 - ws events on update scheduled posts

* MM-60147 - ws events on delete scheduled posts (#28872)

* MM-60147 - ws events on delete scheduled posts

* remove unnecessary channelID

* remove unnecesary memoization
2024-11-05 14:24:50 +01:00
Harshil Sharma
e281b3f37e Feature scheduled messages (#28932)
* Create scheduled post api (#27920)

* Added migration files for Postgres

* Added migrations for MySQL

* Added store method

* Added API and store tests

* Renamed migration after syncing with master

* Added app layer tests

* API is ready

* API is ready

* API is ready

* Renamed migration after syncing with master

* Updated migration list

* Fixed retry layer tests

* Allowed posts with empty messages

* Review fixes

* Reverted an incorrect change

* Renamed migration and fixed ID assignment

* CI

* Send post button changes (#28019)

* added Split button

* WIP

* Added core menu options

* WIP

* WIP

* WIP

* Handled displaying error in creating scheduled post

* lint fixes

* webapp i18n fix

* Review fixes

* Fixed a webapp test

* A few more fixes

* Removed a duplicate comment

* Scheduled post job (#28088)

* Added the job function

* Added query for fetching scheduled posts for pricessing

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* Reafactoring of scheduled post job

* Lint fixes

* Updated i18n files

* FInishing touches

* Added tests for GetScheduledPosts

* Added tests for PermanentlyDeleteScheduledPosts

* Updated all layer

* Some changes as discussed with team

* Added tests for UpdatedScheduledPost

* Code review refactoring

* Added job test

* MM-60120 - Custom time selection (#28120)

* Added a common date time picker modal and used it for post reminder

* Added a common date time picker modal and used it for post reminderggp

* Added modal for custom schedule time and fixed TZ issue

* WIP

* Removed event from useSubmit hook

* Removed event from useSubmit hook

* Added timezone handling

* fixed type error

* Updated i18n strings

* Minor cleanup

* updated snapshots

* review fixes

* Handled event

* Supported for having a DM thread open in RHS while in a regular channel

* Review fixes

* MM-60136 - Scheduled messages tab (#28133)

* WIP

* WIP

* Created Tabs and Tab wrapper with added styling

* Added API to get scheduled posts

* WIP

* Displated scheduled post count

* i18n fix

* Added tests

* Handled asetting active tab absed on URL:

* Reverted unintended change

* Added API to client ad OpenAPI specs

* Renamed file

* Adding fileinfo to schedule posts

* Partial review fixes

* Made get scheduled post API return posts by teamID

* review fixes

* Moved scheduled post redux code to MM-redux package

* Usedd selector factory

* WIP:

* WIP:

* Lint fix

* Fixed an incorrect openapi spec file

* Removed redundent permission check

* Clreaed scheduled post data on logout

* Removed unused i18n string:

* lint fix

* Render scheduled posts (#28208)

* WIP

* WIP

* Created Tabs and Tab wrapper with added styling

* Added API to get scheduled posts

* WIP

* Displated scheduled post count

* i18n fix

* Added tests

* Handled asetting active tab absed on URL:

* Reverted unintended change

* Added API to client ad OpenAPI specs

* Renamed file

* Created common component for draft list item

* WIP

* WIP

* Adding fileinfo to schedule posts

* Basic rendering

* Added count badge to tabs

* WIP

* Made the Drafts LHS iteam appear if no drafts exist but scheduled posts do

* Fixed icon size

* Partial review fixes

* Made get scheduled post API return posts by teamID

* Handled initial vs team switch load

* Displayed scheduled date in panel header

* Added error message and error indiocator

* WIP

* review fixes

* WIP Adding error reason tag

* Added error codes

* Moved scheduled post redux code to MM-redux package

* Usedd selector factory

* WIP:

* WIP:

* Lint fix

* Fixed an incorrect openapi spec file

* Removed redundent permission check

* Clreaed scheduled post data on logout

* Removed unused i18n string:

* lint fix

* Opened rescheduling modal

* Updated graphic for empty state of schduled post list

* Added delete scheduled post option and modal

* Badge and timezone fix

* WIP:

* Added send now confirmation modal

* lint

* Webapp i18n fix

* Fixed webapp test

* Fixed a bug where DM/GM scheduled posts weren't immideatly showing up in UI

* Minor fixes

* WIP

* Review fixes

* Review fixes

* Optimisations

* Fixed reducer name

* Moment optimizatin

* Updated route check

* MM-60144 - added API to update a scheduled post (#28248)

* WIP

* Added api and ap layer for update scheduled post ̛̦̄

* Added API to OpenAI specs, Go client and TS client

* removed permissio check

* Added tests

* Fixed tests

* Added PreUpdate method on scheduled post model

* MM-60131 - Reschedule post integration (#28281)

* Handled rescheduling post in webapp

* Added error handling

* MM-60146 - Delete scheduled post api (#28265)

* WIP

* Added api and ap layer for update scheduled post ̛̦̄

* Added API to OpenAI specs, Go client and TS client

* removed permissio check

* Added tests

* Fixed tests

* Added PreUpdate method on scheduled post model

* Added delete scheduled post API

* Added API to Go client and OpenAPI specs

* Added API to TS client

* Added tests

* CI

* Rmeoved two incorrect code comments

* MM-60653 - Integrated delete scheduled post API (#28296)

* Integrated delete scheduled apost API

* Lint fix

* Review fixes

* Excluded draft checks from scheduled posts (#28370)

* Excluded draft checks from scheduled posts

* Added a removed todo

* MM-60125 - Scheduled post channel indicator (#28320)

* Integrated delete scheduled apost API

* Lint fix

* Added state for storing scheduled posts by channel ID

* Refactored redux store to store scheudled posts by ID, thens tore IDs everywhere

* Refactored redux store to store scheudled posts by ID, thens tore IDs everywhere

* WIP

* Added scheduled post indiocator

* Handled single and multiple scheudled posts

* Review fixes

* Fixed styling and handled center channel, RHS and threads view

* Lint fix

* i18n fix

* Fixed a cycling dependency

* Lint fix

* Added some more comments

* Updated styling

* Review fixes

* Added common component for remote user time and scheduled post indicator

* Updated scheduled post count

* Minor change

* Moved CSS code around

* Fixed a bug where files in scheduled post didn't show up until refresh (#28359)

---------

Co-authored-by: Daniel Espino García <larkox@gmail.com>

* Scheduled post config (#28485)

* Added config

* Added config on server and webapp side

* Added config check in server and webapp

* Added license check

* Added license check

* Added placeholder help text

* Added license check to job

* Fixed job test

* Review fixes

* Updated English text

* Review fixes

* MM-60118 - Added index on ScheduledPosts table (#28579)

* Added index

* Updated indexes

* Scheduled posts misc fixes (#28625)

* Added detailed logging for scheduled post job

* Limited scheduled posts processing to 24 hours

* Marked old scheduled posts as unable to send

* Added t5ests

* converted some logs to trace level

* Fixed a bug causing error message to show up on deleting a scheduled post in a deleted thread (#28630)

* Fixed scheduled posts link in RHS (#28659)

* Fixed scheduled posts link in RHS

* Review fixes

* Fix permission name in scheduled posts by team (#28580)

* Fix permission name

* fix wording

---------

Co-authored-by: Mattermost Build <build@mattermost.com>

* FIxed width of generic modal header to fix browser channel modal (#28639)

* Only consider error-free scheduled posts for indicator in channel and RHS (#28683)

* Show only errro free scheudled posts in post box indicator

* Fixed a bug to handle no scheduled posts

* Fixed draft and scheudled post UI in mobile view (#28680)

* MM-60873 and MM-60872 - Fixed a bug with updating scheduled posts (#28656)

* Fixed a bug with updating scheduled posts

* Better selectors

* MOved shceuled post message length validation to app layer

* MM-60732 - Scheduled posts channel link now takes you to the first scheduled post in channel/thread in list (#28768)

* Ordered scheudle dposts by schgeudled at nad create at

* Ordered in client

* Added scroll to target

* Removed classname prop

* Fixed tests

* Added doc

* Import fix

* MM-60961 - Fixed a bug where API used incoming create at date for scheduled post (#28703)

* Fixed a bug where API used incoming create at date for scheduled post

* Stopped sending created at value for scheduled post

* MM-60785 - Fixed a bug where scheduled posts of channel we are no longer member of didn't show up (#28637)

* Fixed a bug where scheduled posts of channel we are no longer member of didn't show up

* Added a comment

* CI

* Used data loader to optimise laoding missing channels

* Minor refactoring

* MM-60963 - Added common checks for post and scheduled posts (#28713)

* Added commen checks for post and scheuled posts

* Sanitised scheduled posts

* Fixed tests

* Splitted post checks into app and context functions

* Added checks on scheduiled posts job as well:

* i18n fix

* Fixed a test

* Renamed a func

* removed duplicate check

* Scheduled posts UI fixes (#28828)

* Fixed send button and time picker borders

* Fixed center alignment of time picker

* Removed on for today and tomorrow

* Lint fix

* Date time modal hover state fix

* Badge fix

* Fixed a mnerge issue

* Scheduled Post send now and add schedule on draft (#28851)

* Added send now option on scheduled posts

* Minor refactoring

* WIP

* WIP

* WIP

* Lint fix

* i18n fix

* Snapshot update

* Review fixes

* Scheduled post inline editing (#28893)

* Added send now option on scheduled posts

* Minor refactoring

* WIP

* WIP

* WIP

* Lint fix

* i18n fix

* Snapshot update

* Displayed editing component in scheduled post

* Added handling for updating scheduled post

* Handle events

* Fixed escape key issue in scheudled post editing

* Fixes

* Displayed error message for editing error

* Don't show mention warning

* Handled dev mode (#28918)

* MInor fixes

* client fix

* Fixes

* CI

* Removed dev mode behaviour temperorily (#29008)

---------

Co-authored-by: Daniel Espino García <larkox@gmail.com>
Co-authored-by: Eva Sarafianou <eva.sarafianou@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2024-11-04 11:39:35 +05:30
unified-ci-app[bot]
3049191e2f Update latest minor version to 10.3.0 (#29057)
Automatic Merge
2024-11-01 19:17:11 +01:00