* Moving diagnostics into a service
* Fixing golint checks
* Fixing tests
* Renaming from diagnostics to telemetry
* Adding missing files
* Initializing telemetry earlier in the server startup
* Fixing tests
* Adding a log for the telemetryID initialization error
* Addressing PR review comments
* Fixing merge problem
* Removing some extra Diagnostics mentions
* Making tests pass
* MM-27116: Drain push notification channel before closing
Without this, pending push notifications will not be sent when shutting down
or restarting the server.
* Fix race
* Re-arrange server shutdown
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
This avoids the race of assigning the sema field from the start method
which runs in a goroutine.
The race that happens is
==================
WARNING: DATA RACE
Write at 0x00c003a9d1b0 by goroutine 67:
github.com/mattermost/mattermost-server/v5/app.(*PushNotificationsHub).start()
/home/agniva/mattermost/mattermost-server/app/notification_push.go:264 +0x90
Previous read at 0x00c003a9d1b0 by goroutine 69:
github.com/mattermost/mattermost-server/v5/app.(*Server).createPushNotificationsHub()
/home/agniva/mattermost/mattermost-server/app/notification_push.go:260 +0x1d5
* Removing some other fake apps
* More FakeApp removed
* Removing entirely FakeApp
* Fixing some tests
* Fixing get Cluster id from get plugin status
* Fixing failing tests
* Fixing tests
* Fixing test initialization for web
* Fixing InitServer for server tests
* Fixing InitServer for server tests
* Reverting go.sum and go.mod
* Removing unneded HTMLTemplates function in App layer
* Moving back some functions to its old place to easy the review
* Moving back some functions to its old place to easy the review
* Using the last struct2interface version
* Generating store layers
* Fixing merge problems
* Addressing PR comments
* Small fix
* Fixing app tests build
* Fixing tests
* fixing tests
* Fix tests
* Fixing tests
* Fixing tests
* Fixing tests
* Moving license to server struct
* Adding some fixes to the test compilation
* Fixing cluster and some jobs initialization
* Fixing some license tests compilation problems
* Fixing recursive cache invalidation
* Regenerating app layers
* Fix test compilation
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
Before this, for every single request we would create a new http.Client
from scratch. This was costly because every new client has internal
TCP connections which it reuses.
Therefore, originally, beyond a throughput of 800-1200 rps with a
concurrency of 50, beyond which it would run out of file descriptors.
Now it can support upto 8000-10000 rps.
An improvement by a factor of 10.
Throughput numbers:
BenchmarkPushNotification
BenchmarkPushNotification: notification_push_test.go:1478: throughput: 11445.091601 reqs/s
BenchmarkPushNotification: notification_push_test.go:1478: throughput: 9190.761518 reqs/s
BenchmarkPushNotification: notification_push_test.go:1478: throughput: 8334.394764 reqs/s
BenchmarkPushNotification-8 219 5999304 ns/op 3369224 B/op 42127 allocs/op
BenchmarkPushNotification: notification_push_test.go:1478: throughput: 9590.399780 reqs/s
BenchmarkPushNotification: notification_push_test.go:1478: throughput: 9347.137463 reqs/s
BenchmarkPushNotification: notification_push_test.go:1478: throughput: 9053.784250 reqs/s
BenchmarkPushNotification-8 223 5522636 ns/op 3368470 B/op 42034 allocs/op
BenchmarkPushNotification: notification_push_test.go:1478: throughput: 10250.783365 reqs/s
BenchmarkPushNotification: notification_push_test.go:1478: throughput: 9123.292899 reqs/s
BenchmarkPushNotification: notification_push_test.go:1478: throughput: 9173.553180 reqs/s
This is the aggregate count using benchstat.
The number is for a single run of sending 50 requests.
name time/op
PushNotification-8 5.61ms ± 7%
name alloc/op
PushNotification-8 3.36MB ± 1%
name allocs/op
PushNotification-8 42.0k ± 1%
Tested on a machine with ulimit -n = 4096.
When push notification setting was selected to "All Activity", we were
just sending the unread count for the channel from where the message was sent
and not for all channels.
After discussion with @enahum and @migbot, we decided to keep the badge count
to only refer to mentions and not unread posts.
Therefore, we only take the unread count for the user irrespective of the
notify_props settings.
Updated the tests to reflect that.
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* Consistent license message for all the go files
* Fixing the last set of unconsistencies with the license headers
* Addressing PR review comments
* Fixing busy.go and busy_test.go license header
* Update BuildPushNotificationMessage (#12974)
* Update BuildPushNotificationMessage
* Included Category, Version, and Type
* Remove unnecessary function args
* [MM-16232] Build and return fetched notification (#13085)
* Build and return fetched notification
* Remove check on PushNotificationContents
* Ran i18n-extract
* Get UserId from session
* Re-add ReturnStatusOK
* Remove UserId from PushNotificationAck
* Always include channel name
* [MM-16232] Return after writing response and add default message (#13127)
* Return after writing response and add default message
* Include channel ID as well
* Localize default message
* Fix i18n
* Add different types for different mentions
* Remove redundant THREAD_ANY and THREAD_ROOT constants
* Make PostStore.Get return thread in order
* MM-17071 Add initial version of countMentionsFromPost
* MM-17071 Refactor comment mention counting
* MM-17071 Use mention counting when marking post as unread
* Fix shadowing in tests
* Remove repeated check of user count
* Refactor code using MentionType
* Update comments around -1 return value
* Move inner functions out of countMentionsFromPost
* Remove preconditions check as separate test case
* Update comments
* Add User.GetMentionKeys
* Revert "Make PostStore.Get return thread in order"
This reverts commit 22aa010cee359655fe75e1dc899cf0ffb0943c2a.
* Fix tests
* Fix merge conflict
* Add store.MentionAllPosts
* modify Get method in preference_store
* modify interface to match changes on preference store Get method
* modify variable initialization
* fix behavior on GetPreferenceByCategoryAndNameForUser
* Remove NotificationRegistry table and use structured logging
* Fix ackId for notification sent
* Notification logger at server level
* Remove unused i18n strings
* MM-14289 Add Push notification acknowledge identifier and store tracing logs
* MM-14884 include SenderName property in Push Notifications
* Remove @ sign from channel Name in push notifications
* Fix i18n
* Fix push notification model
* fix TestPostNotificationGetChannelName
* Remove colon from model constant
* Fix Notification Registry tests
* Make postId optional for clear notifications
* Update http status when service is not available
Co-Authored-By: enahum <nahumhbl@gmail.com>
* remove @ from push notifications
* prepend @ when display name is set to username
* fix @ in username
* fix username getting prefixed in push notifications
This change is being made to address an issue where the go-i18n
translation library would result in partial-translations when a
given language dictionary was missing a given plural keyword. The
improvement made here leads the translation library to try an
'other' keyword lookup if the first plural keyword fails to have
a value.
This change was not accepted upstream due to concern regarding
changing the behavior, so we are using a fork at this time to
address the issue.
* MM 10658 Change config fields to pointers (#8898)
* Change fields of config structs to pointers and set defaults
MM-10658 https://github.com/mattermost/mattermost-server/issues/8841
* Fix tests that go broken during switching config structs to pointers
MM-10658 https://github.com/mattermost/mattermost-server/issues/8841
* Apply changes of current master while switching config structs to pointers
MM-10658 https://github.com/mattermost/mattermost-server/issues/8841
* Fix new config pointer uses
* Fix app tests
* Fix mail test
* remove debugging statement
* fix TestUpdateConfig
* assign config consistently
* initialize AmazonS3Region in TestS3TestConnection
* initialize fields for TestEmailTest
* fix TestCheckMandatoryS3Fields
We have constants for (most) of the notify props keys. We should use
them consistently, instead of sometimes and other times having strings
scattered around the place.
* Moving goroutine pool
* Auto refactor
* Moving plugins.
* Auto refactor
* Moving fields to server
* Auto refactor
* Removing siteurl duplication.
* Moving reset of app fields
* Auto refactor
* Formatting
* Moving niling of Server to after last use
* Fixing unit tests.
* MM-7188: Cleaning push notification on every read, not only on channel switch
* Removed unnecesary goroutine
* Fixing tests
* Applying suggestion from PR
