* [MM-59089] Add a compliance export constant (#27919)
* add a useful constant
* i18n
* another constant
* another i18n
* [MM-60422] Add GetChannelsWithActivityDuring (#28301)
* modify GetUsersInChannelDuring to accept a slice of channelIds
* add GetChannelsWithActivityDuring
* add compliance export progress message; remove unused custom status
* linting
* tests running too fast
* add batch size config settings
* add store tests
* linting
* empty commit
* i18n changes
* fix i18n ordering
* MM-60570 - Server-side changes consolidating the export CLI with server/ent code (#28640)
* add an i18n field; add the CLI's export directory
* int64 -> int
* Add UntilUpdateAt for MessageExport and AnalyticsPostCount
to merge
* remove now-unused i18n strings
* add TranslationsPreInitFromBuffer to allow CLI to use i18n
* use GetBuilder to simplify; rename TranslationsPreInitFromFileBytes
* [MM-59089] Improve compliance export timings (#1733 - Enterprise repo)
* MM-60422 - Performance and logic fixes for Compliance Exports (#1757 - Enterprise repo)
* MM-60570 - Enterprise-side changes consolidating the export CLI with server/ent code (#1769 - Enterprise repo)
* merge conflicts; missed file from ent branch
* MM-61038 - Add an option to sqlstore.New (#28702)
remove useless comment
add test
add an option to sqlstore.New
* MM-60976: Remove RunExport command from Mattermost binary (#28805)
* remove RunExport command from mattermost binary
* remove the code it was calling
* fix i18n
* remove test (was only testing license, not functionality)
* empty commit
* fix flaky GetChannelsWithActivityDuring test
* MM-60063: Dedicated Export Filestore fix, redo of #1772 (enterprise) (#28803)
* redo filestore fix#1772 (enterprise repo) on top of MM-59966 feature
* add new e2e tests for export filestore
* golint
* ok, note to self: shadowing bad, actually (when there's a defer)
* empty commit
* MM-61137 - Message export: Support 7.8.11 era dbs (#28824)
* support 7.8.11 era dbs by wrapping the store using only what we need
* fix flaky GetChannelsWithActivityDuring test
* add a comment
* only need to define the MEFileInfoStore (the one that'll be overridden)
* blank commit
* MM-60974 - Message Export: Add performance metrics (#28836)
* support 7.8.11 era dbs by wrapping the store using only what we need
* fix flaky GetChannelsWithActivityDuring test
* add a comment
* only need to define the MEFileInfoStore (the one that'll be overridden)
* performance metrics
* cleanup unneeded named returns
* blank commit
* MM-60975 - Message export: Add startTime and endTime to export folder name (#28840)
* support 7.8.11 era dbs by wrapping the store using only what we need
* fix flaky GetChannelsWithActivityDuring test
* add a comment
* only need to define the MEFileInfoStore (the one that'll be overridden)
* performance metrics
* output startTime and endTime in export folder
* empty commit
* merge conflict
* MM-60978 - Message export: Improve xml fields; fix delete semantics (#28873)
* support 7.8.11 era dbs by wrapping the store using only what we need
* fix flaky GetChannelsWithActivityDuring test
* add a comment
* only need to define the MEFileInfoStore (the one that'll be overridden)
* performance metrics
* output startTime and endTime in export folder
* empty commit
* add xml fields, omit when empty, tests
* fix delete semantics; test (and test for update semantics)
* clarify comments
* simplify edited post detection, now there's no edge case.
* add some spacing to help fast running tests
* merge conflicts/updates needed for new deleted post semantics
* linting; fixing tests from upstream merge
* use SafeDereference
* linting
* stronger typing; better wrapped errors; better formatting
* blank commit
* goimports formatting
* fix merge mistake
* minor fixes due to changes in master
* MM-61755 - Simplifying and Support reporting to the db from the CLI (#29281)
* finally clean up JobData struct and stringMap; prep for CLI using db
* and now simplify using StringMapToJobDataWithZeroValues
* remove unused fn
* create JobDataExported; clean up errors
* MM-60176 - Message Export: Global relay cleanup (#29168)
* move global relay logic into global_relay_export
* blank commit
* blank commit
* improve errors
* MM-60693 - Refactor CSV to use same codepath as Actiance (#29191)
* move global relay logic into global_relay_export
* blank commit
* refactor (and simplify) ExportParams into shared
* blank commit
* remove unused fn
* csv now uses pre-calculated joins/leaves like actiance
* improve errors
* remove nil post check; remove ignoredPosts metric
* remove unneeded copy
* MM-61696 - Refactor GlobalRelay to use same codepath as Actiance (#29225)
* move global relay logic into global_relay_export
* blank commit
* refactor (and simplify) ExportParams into shared
* blank commit
* remove unused fn
* csv now uses pre-calculated joins/leaves like actiance
* remove newly unneeded function and its test. goodbye.
* refactor GetPostAttachments for csv + global relay to share
* refactor global_relay_export and fix tests (no changes to output)
* improve errors
* remove nil post check; remove ignoredPosts metric
* remove unneeded copy
* remove unneeded nil check
* PR comments
* MM-61715 - Generalize e2e to all export types 🤖 (#29369)
* move global relay logic into global_relay_export
* blank commit
* refactor (and simplify) ExportParams into shared
* blank commit
* remove unused fn
* csv now uses pre-calculated joins/leaves like actiance
* remove newly unneeded function and its test. goodbye.
* refactor GetPostAttachments for csv + global relay to share
* refactor global_relay_export and fix tests (no changes to output)
* improve errors
* remove nil post check; remove ignoredPosts metric
* remove unneeded copy
* remove unneeded nil check
* PR comments
* refactor isDeletedMsg for all export types
* fix start and endtime, nasty csv createAt bug; bring closer to Actiance
* align unit tests with new logic (e.g. starttime / endtime)
* refactor a TimestampConvert fn for code + tests
* bug: pass templates to global relay (hurray for e2e tests, otherwise...)
* add global relay zip to allowed list (only for tests)
* test helpers
* new templates for e2e tests
* e2e tests... phew.
* linting
* merge conflicts
* unexport PostToRow; add test helper marker
* cleanup, shortening, thanks to PR comments
* MM-61972 - Generalize export data path - Actiance (#29399)
* extract and generalize the export data generation functions
* finish moving test (bc of previous extraction)
* lift a function from common -> shared (to break an import cycle)
* actiance now takes general export data, processes it into actiance data
* bring tests in line with correct sorting rules (upadateAt, messageId)
* fixups, PR comments
* turn strings.Repeat into a more descriptive const
amended: one letter fix; bad rebase
* MM-62009 - e2e clock heisenbug (#29434)
* consolidate assertions; output debuggable diffs (keeping for future)
* refactor test output generator to generators file
* waitUntilZeroPosts + pass through until to job = fix all clock issues
* simplify messages to model.NewId(); remove unneeded waitUntilZeroPosts
* model.NewId() -> storetest.NewTestID()
* MM-61980 - Generalize export data path - CSV (#29482)
* simple refactoring
* increase sleep times for (very) rare test failures
* add extra information to the generic export for CSV
* adj Actiance to handle new generic export (no difference in its output)
* no longer need mergePosts (yay), move getJoinLeavePosts for everyone
* adjust tests for new csv semantics (detailed in summary)
* and need to add the new exported data to the export_data_tests
* rearrange csv writing to happen after data export (more logical)
* linting
* remove debug statements
* figured out what was wrong with global relay e2e test 3; solid now
* PR comments
* MM-61718 - Generalize export data path - Global Relay (#29508)
* move global relay over to using the generalized export data
* performance pass -- not much can be done
* Update server/enterprise/message_export/global_relay_export/global_relay_export.go
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
---------
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
* MM-62058 - Align CSV with Actiance (#29551)
* refactoring actiance files and var names for clarity
* bug found in exported attachments (we used to miss some start/ends)
* changes needed for actiance due to new generic exports
* bringing CSV up to actiance standards
* fixing global relay b/c of new semantics (adding a note on an edge case)
* aligning e2e tests, adding comments to clarify what is expected/tested
* necessary changes; 1 more test for added functionality (ignoreDeleted)
* comment style
* MM-62059 - Align Global Relay with Actiance/CSV; many fixes (#29665)
* core logic changes to general export_data and the specific export paths
* unit tests and e2e tests, covering all new edge cases and all logic
* linting
* better var naming, const value, and cleaning up functions calls
* MM-62436 - Temporarily skip cypress tests that require download link (#29772)
---------
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
* Add metrics for mobile versions snapshots
* Add notifications disabled and fix lint
* Address feedback
* Verify all references to JobTypeActiveUsers
* Fix typos
* Improve platform values
* Add test and MySQL support
* WIP
* Added rebuild channels index functionality
* Added rough logic to send message to all sysadmins
* WIP
* WIP
* WIP
* Cleanup
* i18n fix
* reading through all pages of system admins
* Fixed webapp style
* i18n fix
* Added help text
* i18n fix
* i18n update
* Updated system console button action
* Updated snapshots
* some cleanup
* Updated snapshot
* Update server/channels/app/server.go
Co-authored-by: Daniel Espino García <larkox@gmail.com>
* fixed typo
* Refactoring to improve readibility
* moved index check to API later during config update
* Added some docs
* Updated get system bot
---------
Co-authored-by: Daniel Espino García <larkox@gmail.com>
* [MM-53428] Delete empty drafts on upsert
* Add migrations to fix existing drafts
* Fix CI
* Delete empty drafts entirely from the DB
* Fix lint
* Implement batch migration for deleting drafts
* Missing store layers
* Add updated mock
* Remove unnecessary test
* PR feedback
* Add check for cluster migration
* Fix MySQL
* Don't check for len<2
* Bit of PR feedback
* Use query builder for parameters
* PR feedback
* More PR feedback
* Merge'd
* unit test GetLastCreateAtAndUserIdValuesForEmptyDraftsMigration
* simplified builder interface
* fix DeleteEmptyDraftsByCreateAtAndUserId for MySQL
* rework as batch migration worker
* fix typo
* log ip address on version mismatches too
* simplify reset semantics
* remove trace log in favour of low spam
* document parameters for clarity
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
* pass a logger instead of embedding on job
* leverage mlog.Millis
* use worker logger with HandleJobPanic
* rely on existing LogClone instead
* guard Job.LogClone against nil Job
* s/workername/worker_name
* Revert "rely on existing LogClone instead"
This reverts commit 17303cbac90d4b01815abca1309b78b97de368fb.
* Revert "guard Job.LogClone against nil Job"
This reverts commit f1ae22dee58d76f084582857830ffe8d4c546d7e.
* [MM-37984] Allow Desktop App to authenticate via external providers outside of the app on supported servers
* PR feedback
* Add support for mattermost-dev protocol for development use
* Update server/channels/db/migrations/postgres/000110_create_desktop_tokens.up.sql
* Fix silly typo
* Update server/channels/db/migrations/postgres/000110_create_desktop_tokens.up.sql
* Remove storage of client token, only validate it on the client
* Update migrations
* Add concurrently create index
* Remove CONCURRENTLY for now
* Fix issue with changing history
* Remove old migration
* Use idempotent statement to drop old index
* Remove reference to old table
Bifrost now encodes all image paths. Due to this
one-way translation, we need to encode all the older
image paths as well.
After this is done, we can remove the double-lookup.
https://mattermost.atlassian.net/browse/MM-53747
```release-note
NONE
```
Co-authored-by: Mattermost Build <build@mattermost.com>
* WIP
* Add rate limiting for desktop token API
* Missing mocks
* Style fixes
* Update snapshots
* Maybe use an actual redirect link :P
* Refactoring for tests
* Add tests for server
* Fix lint issue
* Fix tests
* Fix lint
* Add front-end screen component
* Component logic
* Style changes
* Quick style fix
* Lint fixes
* Initial PR feedback
* Enable logging into the browser as well when completing the login process
* Refactor to push more logic to the other component
* Remove unnecessary helper code
* Fix i18n
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* MM-46410: adds urgency on mention counts
We have introduced priority for posts in
https://github.com/mattermost/mattermost-webapp/pull/10951.
We do need to color the mention badges in the webapp with a prominent
color when a mention is posted in an urgent message.
A thread has urgent mentions if the root post is marked as urgent, and
the replies contain mentions to the user viewing the thread.
This PR adds two columns, urgentmentioncount, and isurgent, in
channelmembers, and threads tables respectively.
Furthermore when asking for team/thread mention counts, we also return
urgent mention counts for the user.
* Adds PostAcknowledgements table and apis
* job init and fetch mentions
* add-migrations
* delete-expired
* send-notifications
* Fetches post priority in batches
* stop-notifications
* stop-notification-on-reply
* MM-47750: Adds PostAcknowledgements table and apis
- Adds post acknowledgement api/app/store methods to be able to save and
delete post acknowledgements by users.
- Adds wesbsocket events for acknowledgement created/deleted
- Returns post acknowledgements in the post's metadata
* add-license-check
* add-pagination
* delete on channel and team
* validate guests
* add configs
* move create priority post check from app to api
* Add desktop notifications
* check status
* use config in job
* add IsUrgent check
* Add last-sent-at
* validate max recipients
* Update lastSentAt
* Validate min. recipient
* send email notification only once
* remove email notifications
* use latest time from config to run job
* Add notifications counter
* publish events to mentioned users only
* pickup license updates in scheduler
* don't allow post owner to stop notifications
* follow normal notifications behaviour
* Validates persistent notifications interval
* move logic of handling valid and expired posts into sql
* Adds persistent notifications in the webapp
---------
Co-authored-by: koox00 <3829551+koox00@users.noreply.github.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
