* Working on refactoring jobs service
* Making more consistent with the previous existing code
* Remove no longer needed functions
* Making a base PeridicScheduler to use it in most of the schedulers implementations
* Removing accidental complexity from on of the jobs
* Removing accidental complexity from expirynotify
* Fixing compilation from previous commit
* Remove accidental complexity from the export_delete job
* Simplifying the workers by making a reusable worker
* Using simple worker for export_delete job
* Simpliying export process job
* Simpliying extract content job
* Simpliying import delete job
* Simpliying import process job
* Simpliying product noticies job
* Simpliying fix crt channel unreads job (only removing the uneeded register function)
* Simpliying migrations job (only removing the uneeded register function)
* fixup
* Simpliying plugins job (only removing the uneeded register function)
* Simpliying bleve indexing job (only removing the uneeded register function)
* Simpliying resend invitation email job (only removing the uneeded register function)
* Fixing tests
* Simplifying migration tests infrastructure
* Adding missed license to files
* Adding an empty file to imports package to ensure this package exist even without enterprise repo
* Regenerating einterfaces mocks
* Adding missed license to files
* Updating i18n/en.json file
* help fixing enterprise tests compilation
* Adding new DailyScheduler
* Fixing typo and changing the waitTime type for periodic sechduler
* Making the daily scheduler more generic
* Adding comments to clarify not used parameters in interface scheduler interface implementations
* Using merror to handle multiple errors in jobs workers
* Fixing linter errors
* Addressing PR review comments
* Reverting go.tools.mod changes
* Removing the static check for worker type in the model (moving it to the insertion of new jobs
* Moving migrations job to the jobs directory
* Fixing (and improving a bit) tests
* Apply suggestions from code review
Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
* Fixing enterprise tests
* Removing unneeded InitWorkers/InitSchedulers calls
* Fix expirenotify job when error happens
* Fixing govet errors
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
This corrects a bug where exports that contained deleted teams
could fail to import. Channels must be skipped that belong to these
deleted teams in order to produce a valid import bundle.
* Fix improper attachments in replies
* Fix import data path
* Improve errors
* Fix importing attachments directly from zip file
* Add some test cases to cover error paths
* app/import: do not break import process if a dm channel is invalid
* update test headers
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-30882: Fix read-after-write issue for demoting user
In (*App).DemoteUserToGuest, we would demote a user, and then immediately
read it back to do future operations from the user. This reading back
of the user had the effect of sticking the old value into the cache
after which it would never be updated.
There was another issue along with this, which was when the invalidation
message would broadcast across the cluster, it would hit the cache invalidation
problem where an unrelated store call would miss the cache because
it was invalidated, and then again read from replica and stick the old value.
To fix all these, we return the new value directly from the store method
to avoid having the app to read it again.
And we add a map in the localcache layer which tracks invalidations made,
and then switch to use master if it's true.
The core change is fairly limited, but due to changing the store method signatures,
a lot of code needed to be updated to pass "context.Background". Therefore the PR
just "appears" to be big, but the main changes are limited to app/user.go,
sqlstore/user_store.go and user_layer.go
https://mattermost.atlassian.net/browse/MM-30882
```release-note
Fix an issue where demoting a user to guest would not take effect in
an environment with read replicas.
```
* Fix concurrent map access
* Fixing mistakes
* fix tests
* Include filepaths for post attachments
* Cleanup
* Enable exporting file attachments
* Fix file import
* Enable zip export
* Support creating missing directories when unzipping
* Add test
* Add translations
* Export direct channel posts attachments
* Fix returned values order
Remove pointer to slice in return
* [MM-31597] Implement export process job (#16626)
* Implement export process job
* Add translations
* Remove unused value
* [MM-31249] Add /exports API endpoint (#16633)
* Implement API endpoints to list, download and delete export files
* Add endpoint for single resource
* Update i18n/en.json
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* Update i18n/en.json
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* Fix var name
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* format using `goimports -local github.com/mattermost/mattermost-server/v5 -w`
* added goimports lint check to .golangci.yml
* format using `goimports -local github.com/mattermost/mattermost-server/v5 -w` for a corner case
* make app-layers, *-mocks and store-layers for ci check
Co-authored-by: Mahmudul Haque <mahmudulhaque@protonmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Test added to check support for import of messages sent to your own DM
channel
* Support for single member direct channel messages and posts
Now bulk import is able to export/import the self direct channel and
the posts inside
* Improve how the copy of channel members for DM and DM posts
* Improved the tests for the export self-channels
* Consistent license message for all the go files
* Fixing the last set of unconsistencies with the license headers
* Addressing PR review comments
* Fixing busy.go and busy_test.go license header
* Migrating audit store
* Final migration example for the audit store
* async example
* Ending migration
* Removing Async helper
* Fixing tests
* Fixing govet problems with the StoreResult instanstiation
* transplant the existing PR into the working tree
* start addressing review comments
* move existing direct channel export code into this branch
* modify channel exporter to use squirell and populate members in two steps
* use squirrel to build sql queries for channel and dm/gm export methods
* remove debug helpers and use Username instead of UserId
* unit test for DM Channel exporter
* add more unit tests for channel export
* add test for DM/GM post export
* checkpoint with failing test for postgres
* use getQueryBuilder to make sure squirrel uses the correct formatting
for each database
* add a test for post export
* fix shadowed vars that broke the build
* address review comments and add tests to support it
* address review comments and add a mlog call
* s/Info/Debug/
* address review comments in post_store
* address review comments in channel_store
* address review comments in export
* address review comment in post_store: drop GroupBy
* address review comment on supplier: move getQueryBuilder to sqlstore
* address review comments: explicit TearDown
* address review comments: improve test coverage
* address review comments: make sure public and private channels are excluded
* address review comments: improve test coverage
* address review comments: make sure Channels table gets truncated after
each test
* more cleanups and better assertions
* wrap PostStore in a StoreTestWithSqlSupplier
* last minute changes: improve post export test coverage and check members
* address review comments: make sure all posts have their channel
members set
* address review comments: make sure all posts have their ChannelMembers
exported correctly
* gofmt fix
* sort channels so it's possible to assert on index
* [MM-12667] Allow including deactivated users in bulk import and export
1. Added `deleteAt` for user type import line
2. Adding deactivated users data in bulk export
3. Importing deactivated users data in bulk import
Added/Updated relevant test cases
* Fixed export of replies to posts by deleted users.
Updated tests for same
* First version of the preference exporter
* First working version of the preference export
* Removed useless debug line
* Added special case of empty string to harverst the omitempty feature
* Removed dangling line from rebase
* Moved exportablepreference in scope
* Fix go vet issue
* Fix formating
* Add simple test
* Add full test for BulkExportUsers
* Improve variable naming
* Added capability to export reactions of posts
* Added capability to export reactions of replies of a post
* Added test case to test the reactions of a post
