* [MM-59089] Add a compliance export constant (#27919)
* add a useful constant
* i18n
* another constant
* another i18n
* [MM-60422] Add GetChannelsWithActivityDuring (#28301)
* modify GetUsersInChannelDuring to accept a slice of channelIds
* add GetChannelsWithActivityDuring
* add compliance export progress message; remove unused custom status
* linting
* tests running too fast
* add batch size config settings
* add store tests
* linting
* empty commit
* i18n changes
* fix i18n ordering
* MM-60570 - Server-side changes consolidating the export CLI with server/ent code (#28640)
* add an i18n field; add the CLI's export directory
* int64 -> int
* Add UntilUpdateAt for MessageExport and AnalyticsPostCount
to merge
* remove now-unused i18n strings
* add TranslationsPreInitFromBuffer to allow CLI to use i18n
* use GetBuilder to simplify; rename TranslationsPreInitFromFileBytes
* [MM-59089] Improve compliance export timings (#1733 - Enterprise repo)
* MM-60422 - Performance and logic fixes for Compliance Exports (#1757 - Enterprise repo)
* MM-60570 - Enterprise-side changes consolidating the export CLI with server/ent code (#1769 - Enterprise repo)
* merge conflicts; missed file from ent branch
* MM-61038 - Add an option to sqlstore.New (#28702)
remove useless comment
add test
add an option to sqlstore.New
* MM-60976: Remove RunExport command from Mattermost binary (#28805)
* remove RunExport command from mattermost binary
* remove the code it was calling
* fix i18n
* remove test (was only testing license, not functionality)
* empty commit
* fix flaky GetChannelsWithActivityDuring test
* MM-60063: Dedicated Export Filestore fix, redo of #1772 (enterprise) (#28803)
* redo filestore fix#1772 (enterprise repo) on top of MM-59966 feature
* add new e2e tests for export filestore
* golint
* ok, note to self: shadowing bad, actually (when there's a defer)
* empty commit
* MM-61137 - Message export: Support 7.8.11 era dbs (#28824)
* support 7.8.11 era dbs by wrapping the store using only what we need
* fix flaky GetChannelsWithActivityDuring test
* add a comment
* only need to define the MEFileInfoStore (the one that'll be overridden)
* blank commit
* MM-60974 - Message Export: Add performance metrics (#28836)
* support 7.8.11 era dbs by wrapping the store using only what we need
* fix flaky GetChannelsWithActivityDuring test
* add a comment
* only need to define the MEFileInfoStore (the one that'll be overridden)
* performance metrics
* cleanup unneeded named returns
* blank commit
* MM-60975 - Message export: Add startTime and endTime to export folder name (#28840)
* support 7.8.11 era dbs by wrapping the store using only what we need
* fix flaky GetChannelsWithActivityDuring test
* add a comment
* only need to define the MEFileInfoStore (the one that'll be overridden)
* performance metrics
* output startTime and endTime in export folder
* empty commit
* merge conflict
* MM-60978 - Message export: Improve xml fields; fix delete semantics (#28873)
* support 7.8.11 era dbs by wrapping the store using only what we need
* fix flaky GetChannelsWithActivityDuring test
* add a comment
* only need to define the MEFileInfoStore (the one that'll be overridden)
* performance metrics
* output startTime and endTime in export folder
* empty commit
* add xml fields, omit when empty, tests
* fix delete semantics; test (and test for update semantics)
* clarify comments
* simplify edited post detection, now there's no edge case.
* add some spacing to help fast running tests
* merge conflicts/updates needed for new deleted post semantics
* linting; fixing tests from upstream merge
* use SafeDereference
* linting
* stronger typing; better wrapped errors; better formatting
* blank commit
* goimports formatting
* fix merge mistake
* minor fixes due to changes in master
* MM-61755 - Simplifying and Support reporting to the db from the CLI (#29281)
* finally clean up JobData struct and stringMap; prep for CLI using db
* and now simplify using StringMapToJobDataWithZeroValues
* remove unused fn
* create JobDataExported; clean up errors
* MM-60176 - Message Export: Global relay cleanup (#29168)
* move global relay logic into global_relay_export
* blank commit
* blank commit
* improve errors
* MM-60693 - Refactor CSV to use same codepath as Actiance (#29191)
* move global relay logic into global_relay_export
* blank commit
* refactor (and simplify) ExportParams into shared
* blank commit
* remove unused fn
* csv now uses pre-calculated joins/leaves like actiance
* improve errors
* remove nil post check; remove ignoredPosts metric
* remove unneeded copy
* MM-61696 - Refactor GlobalRelay to use same codepath as Actiance (#29225)
* move global relay logic into global_relay_export
* blank commit
* refactor (and simplify) ExportParams into shared
* blank commit
* remove unused fn
* csv now uses pre-calculated joins/leaves like actiance
* remove newly unneeded function and its test. goodbye.
* refactor GetPostAttachments for csv + global relay to share
* refactor global_relay_export and fix tests (no changes to output)
* improve errors
* remove nil post check; remove ignoredPosts metric
* remove unneeded copy
* remove unneeded nil check
* PR comments
* MM-61715 - Generalize e2e to all export types 🤖 (#29369)
* move global relay logic into global_relay_export
* blank commit
* refactor (and simplify) ExportParams into shared
* blank commit
* remove unused fn
* csv now uses pre-calculated joins/leaves like actiance
* remove newly unneeded function and its test. goodbye.
* refactor GetPostAttachments for csv + global relay to share
* refactor global_relay_export and fix tests (no changes to output)
* improve errors
* remove nil post check; remove ignoredPosts metric
* remove unneeded copy
* remove unneeded nil check
* PR comments
* refactor isDeletedMsg for all export types
* fix start and endtime, nasty csv createAt bug; bring closer to Actiance
* align unit tests with new logic (e.g. starttime / endtime)
* refactor a TimestampConvert fn for code + tests
* bug: pass templates to global relay (hurray for e2e tests, otherwise...)
* add global relay zip to allowed list (only for tests)
* test helpers
* new templates for e2e tests
* e2e tests... phew.
* linting
* merge conflicts
* unexport PostToRow; add test helper marker
* cleanup, shortening, thanks to PR comments
* MM-61972 - Generalize export data path - Actiance (#29399)
* extract and generalize the export data generation functions
* finish moving test (bc of previous extraction)
* lift a function from common -> shared (to break an import cycle)
* actiance now takes general export data, processes it into actiance data
* bring tests in line with correct sorting rules (upadateAt, messageId)
* fixups, PR comments
* turn strings.Repeat into a more descriptive const
amended: one letter fix; bad rebase
* MM-62009 - e2e clock heisenbug (#29434)
* consolidate assertions; output debuggable diffs (keeping for future)
* refactor test output generator to generators file
* waitUntilZeroPosts + pass through until to job = fix all clock issues
* simplify messages to model.NewId(); remove unneeded waitUntilZeroPosts
* model.NewId() -> storetest.NewTestID()
* MM-61980 - Generalize export data path - CSV (#29482)
* simple refactoring
* increase sleep times for (very) rare test failures
* add extra information to the generic export for CSV
* adj Actiance to handle new generic export (no difference in its output)
* no longer need mergePosts (yay), move getJoinLeavePosts for everyone
* adjust tests for new csv semantics (detailed in summary)
* and need to add the new exported data to the export_data_tests
* rearrange csv writing to happen after data export (more logical)
* linting
* remove debug statements
* figured out what was wrong with global relay e2e test 3; solid now
* PR comments
* MM-61718 - Generalize export data path - Global Relay (#29508)
* move global relay over to using the generalized export data
* performance pass -- not much can be done
* Update server/enterprise/message_export/global_relay_export/global_relay_export.go
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
---------
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
* MM-62058 - Align CSV with Actiance (#29551)
* refactoring actiance files and var names for clarity
* bug found in exported attachments (we used to miss some start/ends)
* changes needed for actiance due to new generic exports
* bringing CSV up to actiance standards
* fixing global relay b/c of new semantics (adding a note on an edge case)
* aligning e2e tests, adding comments to clarify what is expected/tested
* necessary changes; 1 more test for added functionality (ignoreDeleted)
* comment style
* MM-62059 - Align Global Relay with Actiance/CSV; many fixes (#29665)
* core logic changes to general export_data and the specific export paths
* unit tests and e2e tests, covering all new edge cases and all logic
* linting
* better var naming, const value, and cleaning up functions calls
* MM-62436 - Temporarily skip cypress tests that require download link (#29772)
---------
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
We cannot simply use Elasticsearch any more because
we support Opensearch as well.
Therefore changing Elasticsearch to just say Search.
Note: There are still some cases where Elasticsearch
is used. That is intentional and necessary.
```release-note
NONE
```
* Add DeleteSidebarCategory method and channel category API tests
* Add test to verify channels move to default categories when custom category is deleted
* rename channel category tests and remove duplicate context
* TestNotifications fix
---------
Co-authored-by: Fume <contact@fumedev.com>
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
Page load is one of the metrics that we track and present
to MLT. However, in its current form, it is not very
actionable because it also contains the network latency.
We split the whole metric into these parts:
startTime
|
responseStart = TTFB
|
responseEnd = TTLB
|
domInteractive = Start of processing phase
|
loadEventEnd = Load complete
This gives us better visibility into exactly
which phase in the load process is slow.
I have experimented with other metrics like
- domContentLoadedEventStart
- domContentLoadedEventEnd
- domComplete
and observed that they do not have sufficient
gaps in the timespan to have any relevance.
Additionally, I have moved TTFB from being a
web vitals metric to being tracked from the performance
metrics to remain consistent with the other navigation
metrics measured.
Lastly, I took this chance to improve some of the
validation errors that we threw to include more
context into the input that was passed and why
does it fail.
This also meant that I had to change the tests
to check for error strings rather than direct
errors which is a bad thing, but I don't think
it's worth the effort trying to have named error
variables for all of them.
https://mattermost.atlassian.net/browse/MM-61886
```release-note
NONE
```
* poc - wip
* add search files across teams
* eslint
* fix existing tests
* fix webapp style
* fix test
* add api doc
* change initial state in test
* add tests on API
* add tests on file info layer
* fix file search tags
* add rhs reducer test
* reset team selected when the RHS is suppressed
* change css to reflect UI
* fix style
* fix doc wording
* make getSearchTeam return currentTeamId when value is not set
* await is unnecessary
* revert boolean check and add test
* add comment to getSearchTeam to let dev knows it defaults to currentTeam
* remove redundant team check
* simplfy test
* fix style check
---------
Co-authored-by: Caleb Roseland <caleb@calebroseland.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
Since we have the auto-retry mechanism for replicas,
there is not much benefit in trying 5 times before deciding
to move on.
In the earlier model, we would fail the server startup,
so it made sense to check as many times as possible.
Also reducing the sleep interval so that we can improve
the boot up time in case of a bad replica.
```release-note
NONE
```
* fix: Add job name to the publish report step
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Antonis Stamatiou <stamatiou.antonis@gmail.com>
* Added support for S3 storage classes
* Added missing translations for S3 storage class
* Changed default storage class values to preserve original behaviour
* Changed storage class description and example
* Fix translations ordering
* Change configuration defaults to empty strings
* Remove redundant empty string check
* Validate storage class variable against constants from S3 API docs
* Validate export storage class against constants
* Use slices for config validation
* Applied patch for translation ordering
* Add test notification tool
* Add frontend styles
* Remove option from admin view
* Refactor create post and add translations
* Fix several CI errors
* Fix API and frontend snapshots
* Refactor trailing and leading icon on buttons
* Add different button states
* i18n-extract
* Fix wrong text
* Add tests
* Fix wrong string
* Fix test
* feat: E2E send test notifications (#28371)
* Refactor send desktop notification
* Address rest of the feedback
* Fix tests
* Add correct link
* Fix test
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: yasserfaraazkhan <attitude3cena.yf@gmail.com>
* Create scheduled post api (#27920)
* Added migration files for Postgres
* Added migrations for MySQL
* Added store method
* Added API and store tests
* Renamed migration after syncing with master
* Added app layer tests
* API is ready
* API is ready
* API is ready
* Renamed migration after syncing with master
* Updated migration list
* Fixed retry layer tests
* Allowed posts with empty messages
* Review fixes
* Reverted an incorrect change
* Renamed migration and fixed ID assignment
* CI
* Send post button changes (#28019)
* added Split button
* WIP
* Added core menu options
* WIP
* WIP
* WIP
* Handled displaying error in creating scheduled post
* lint fixes
* webapp i18n fix
* Review fixes
* Fixed a webapp test
* A few more fixes
* Removed a duplicate comment
* Scheduled post job (#28088)
* Added the job function
* Added query for fetching scheduled posts for pricessing
* WIP
* WIP
* WIP
* WIP
* WIP
* WIP
* Reafactoring of scheduled post job
* Lint fixes
* Updated i18n files
* FInishing touches
* Added tests for GetScheduledPosts
* Added tests for PermanentlyDeleteScheduledPosts
* Updated all layer
* Some changes as discussed with team
* Added tests for UpdatedScheduledPost
* Code review refactoring
* Added job test
* MM-60120 - Custom time selection (#28120)
* Added a common date time picker modal and used it for post reminder
* Added a common date time picker modal and used it for post reminderggp
* Added modal for custom schedule time and fixed TZ issue
* WIP
* Removed event from useSubmit hook
* Removed event from useSubmit hook
* Added timezone handling
* fixed type error
* Updated i18n strings
* Minor cleanup
* updated snapshots
* review fixes
* Handled event
* Supported for having a DM thread open in RHS while in a regular channel
* Review fixes
* MM-60136 - Scheduled messages tab (#28133)
* WIP
* WIP
* Created Tabs and Tab wrapper with added styling
* Added API to get scheduled posts
* WIP
* Displated scheduled post count
* i18n fix
* Added tests
* Handled asetting active tab absed on URL:
* Reverted unintended change
* Added API to client ad OpenAPI specs
* Renamed file
* Adding fileinfo to schedule posts
* Partial review fixes
* Made get scheduled post API return posts by teamID
* review fixes
* Moved scheduled post redux code to MM-redux package
* Usedd selector factory
* WIP:
* WIP:
* Lint fix
* Fixed an incorrect openapi spec file
* Removed redundent permission check
* Clreaed scheduled post data on logout
* Removed unused i18n string:
* lint fix
* Render scheduled posts (#28208)
* WIP
* WIP
* Created Tabs and Tab wrapper with added styling
* Added API to get scheduled posts
* WIP
* Displated scheduled post count
* i18n fix
* Added tests
* Handled asetting active tab absed on URL:
* Reverted unintended change
* Added API to client ad OpenAPI specs
* Renamed file
* Created common component for draft list item
* WIP
* WIP
* Adding fileinfo to schedule posts
* Basic rendering
* Added count badge to tabs
* WIP
* Made the Drafts LHS iteam appear if no drafts exist but scheduled posts do
* Fixed icon size
* Partial review fixes
* Made get scheduled post API return posts by teamID
* Handled initial vs team switch load
* Displayed scheduled date in panel header
* Added error message and error indiocator
* WIP
* review fixes
* WIP Adding error reason tag
* Added error codes
* Moved scheduled post redux code to MM-redux package
* Usedd selector factory
* WIP:
* WIP:
* Lint fix
* Fixed an incorrect openapi spec file
* Removed redundent permission check
* Clreaed scheduled post data on logout
* Removed unused i18n string:
* lint fix
* Opened rescheduling modal
* Updated graphic for empty state of schduled post list
* Added delete scheduled post option and modal
* Badge and timezone fix
* WIP:
* Added send now confirmation modal
* lint
* Webapp i18n fix
* Fixed webapp test
* Fixed a bug where DM/GM scheduled posts weren't immideatly showing up in UI
* Minor fixes
* WIP
* Review fixes
* Review fixes
* Optimisations
* Fixed reducer name
* Moment optimizatin
* Updated route check
* MM-60144 - added API to update a scheduled post (#28248)
* WIP
* Added api and ap layer for update scheduled post ̛̦̄
* Added API to OpenAI specs, Go client and TS client
* removed permissio check
* Added tests
* Fixed tests
* Added PreUpdate method on scheduled post model
* MM-60131 - Reschedule post integration (#28281)
* Handled rescheduling post in webapp
* Added error handling
* MM-60146 - Delete scheduled post api (#28265)
* WIP
* Added api and ap layer for update scheduled post ̛̦̄
* Added API to OpenAI specs, Go client and TS client
* removed permissio check
* Added tests
* Fixed tests
* Added PreUpdate method on scheduled post model
* Added delete scheduled post API
* Added API to Go client and OpenAPI specs
* Added API to TS client
* Added tests
* CI
* Rmeoved two incorrect code comments
* MM-60653 - Integrated delete scheduled post API (#28296)
* Integrated delete scheduled apost API
* Lint fix
* Review fixes
* Excluded draft checks from scheduled posts (#28370)
* Excluded draft checks from scheduled posts
* Added a removed todo
* MM-60125 - Scheduled post channel indicator (#28320)
* Integrated delete scheduled apost API
* Lint fix
* Added state for storing scheduled posts by channel ID
* Refactored redux store to store scheudled posts by ID, thens tore IDs everywhere
* Refactored redux store to store scheudled posts by ID, thens tore IDs everywhere
* WIP
* Added scheduled post indiocator
* Handled single and multiple scheudled posts
* Review fixes
* Fixed styling and handled center channel, RHS and threads view
* Lint fix
* i18n fix
* Fixed a cycling dependency
* Lint fix
* Added some more comments
* Updated styling
* Review fixes
* Added common component for remote user time and scheduled post indicator
* Updated scheduled post count
* Minor change
* Moved CSS code around
* Fixed a bug where files in scheduled post didn't show up until refresh (#28359)
---------
Co-authored-by: Daniel Espino García <larkox@gmail.com>
* Scheduled post config (#28485)
* Added config
* Added config on server and webapp side
* Added config check in server and webapp
* Added license check
* Added license check
* Added placeholder help text
* Added license check to job
* Fixed job test
* Review fixes
* Updated English text
* Review fixes
* MM-60118 - Added index on ScheduledPosts table (#28579)
* Added index
* Updated indexes
* Scheduled posts misc fixes (#28625)
* Added detailed logging for scheduled post job
* Limited scheduled posts processing to 24 hours
* Marked old scheduled posts as unable to send
* Added t5ests
* converted some logs to trace level
* Fixed a bug causing error message to show up on deleting a scheduled post in a deleted thread (#28630)
* Fixed scheduled posts link in RHS (#28659)
* Fixed scheduled posts link in RHS
* Review fixes
* Fix permission name in scheduled posts by team (#28580)
* Fix permission name
* fix wording
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* FIxed width of generic modal header to fix browser channel modal (#28639)
* Only consider error-free scheduled posts for indicator in channel and RHS (#28683)
* Show only errro free scheudled posts in post box indicator
* Fixed a bug to handle no scheduled posts
* Fixed draft and scheudled post UI in mobile view (#28680)
* MM-60873 and MM-60872 - Fixed a bug with updating scheduled posts (#28656)
* Fixed a bug with updating scheduled posts
* Better selectors
* MOved shceuled post message length validation to app layer
* MM-60732 - Scheduled posts channel link now takes you to the first scheduled post in channel/thread in list (#28768)
* Ordered scheudle dposts by schgeudled at nad create at
* Ordered in client
* Added scroll to target
* Removed classname prop
* Fixed tests
* Added doc
* Import fix
* MM-60961 - Fixed a bug where API used incoming create at date for scheduled post (#28703)
* Fixed a bug where API used incoming create at date for scheduled post
* Stopped sending created at value for scheduled post
* MM-60785 - Fixed a bug where scheduled posts of channel we are no longer member of didn't show up (#28637)
* Fixed a bug where scheduled posts of channel we are no longer member of didn't show up
* Added a comment
* CI
* Used data loader to optimise laoding missing channels
* Minor refactoring
* MM-60963 - Added common checks for post and scheduled posts (#28713)
* Added commen checks for post and scheuled posts
* Sanitised scheduled posts
* Fixed tests
* Splitted post checks into app and context functions
* Added checks on scheduiled posts job as well:
* i18n fix
* Fixed a test
* Renamed a func
* removed duplicate check
* Scheduled posts UI fixes (#28828)
* Fixed send button and time picker borders
* Fixed center alignment of time picker
* Removed on for today and tomorrow
* Lint fix
* Date time modal hover state fix
* Badge fix
* Fixed a mnerge issue
* Scheduled Post send now and add schedule on draft (#28851)
* Added send now option on scheduled posts
* Minor refactoring
* WIP
* WIP
* WIP
* Lint fix
* i18n fix
* Snapshot update
* Review fixes
* Scheduled post inline editing (#28893)
* Added send now option on scheduled posts
* Minor refactoring
* WIP
* WIP
* WIP
* Lint fix
* i18n fix
* Snapshot update
* Displayed editing component in scheduled post
* Added handling for updating scheduled post
* Handle events
* Fixed escape key issue in scheudled post editing
* Fixes
* Displayed error message for editing error
* Don't show mention warning
* Handled dev mode (#28918)
* MInor fixes
* client fix
* Fixes
* CI
* Removed dev mode behaviour temperorily (#29008)
---------
Co-authored-by: Daniel Espino García <larkox@gmail.com>
Co-authored-by: Eva Sarafianou <eva.sarafianou@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
This originated from https://github.com/mattermost/mattermost/issues/15249.
However, the original idea was discarded https://github.com/mattermost/mattermost/issues/15249#issuecomment-709713065
as being too complicated to implement. Then I had another
idea to implement it just for session objects.
My thinking was that since every single request allocates a new
session struct, it would be good to use a sync.Pool for that.
However, 4 years later, now we know that the primary bottleneck
in app performance comes from websocket event marshalling.
Therefore, while it would be good to do this, it is difficult
to do it correctly (as shown by the numerous racy tests).
Hence, reverting this.
```release-note
NONE
```
* Revert "MM-54502 - Update regex to force first character to be alpha (#24675)"
This reverts commit 08ed72f060.
* update tests for reverted PR
* Revert "update tests for reverted PR"
This reverts commit c1cf5495fa129008e44a8fcdc0ee34762b33dd34.
* Revert "Revert "MM-54502 - Update regex to force first character to be alpha (#24675)""
This reverts commit 6920b979c8dba86a837bafff7c057dc609c6aa58.
* only revert important parts, not tests
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* Excludes remote channels from channel search
This is done through a new body parameter in the SearchAllChannels
endpoint that allows to search for local only channels, which are
either channels that are shared but marked as homed locally, or
channels that are not shared at all.
* fix lint
* Fix tests
---------
Co-authored-by: Caleb Roseland <caleb@calebroseland.com>
* additional validation for channelbookmark
* add fixes for webapp
* only set permissions correct for type
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
