* Removing some other fake apps
* More FakeApp removed
* Removing entirely FakeApp
* Fixing some tests
* Fixing get Cluster id from get plugin status
* Fixing failing tests
* Fixing tests
* Fixing test initialization for web
* Fixing InitServer for server tests
* Fixing InitServer for server tests
* Reverting go.sum and go.mod
* Removing unneded HTMLTemplates function in App layer
* Moving back some functions to its old place to easy the review
* Moving back some functions to its old place to easy the review
* Using the last struct2interface version
* Generating store layers
* Fixing merge problems
* Addressing PR comments
* Small fix
* Fixing app tests build
* Fixing tests
* fixing tests
* Fix tests
* Fixing tests
* Fixing tests
* Fixing tests
* Moving license to server struct
* Adding some fixes to the test compilation
* Fixing cluster and some jobs initialization
* Fixing some license tests compilation problems
* Fixing recursive cache invalidation
* Regenerating app layers
* Fix test compilation
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
We invert the skipSend condition and outdent the remaining block
to make the code a bit more idiomatic.
While here, we also change the dropping message level from info to
warn because that's what it should be.
* MM-23800: remove goroutineID and stack printing
Each hub has a goroutineID which is calculated with a known hack.
The FAQ clearly explains why goroutines don't have an id:
https://golang.org/doc/faq#no_goroutine_id.
We only added that because sometimes the hub would be deadlocked and
having the goroutineID would be useful when getting the stack trace.
This is also problematic in stress tests because the hubs would
frequently get overloaded and the logs would unnecessarily have stack traces.
But that was in the past, and we have done extensive testing with
load tests and fuzz testing to smooth any rough edges remaining.
Including adding additional metrics for hub buffer size.
Monitoring the metrics is a better way to approach this problem.
Therefore, we remove these kludges from the code.
* Also remove deadlock checking code
There is no need for that anymore since
we are getting rid of the stack printing anyways.
Let's do a wholesale refactor and clean up the codebase.
* MM-23805: Refactor web_hub
This is a beginning of the refactoring of the websocket code.
To start off with, we unexport some methods and constants which did not
need to be exported. There are more remaining but some are out of scope for this PR.
The main chunk of refactor is to unexport the webconn send channel
which was the main cause of panics. Since we were directly sending
to the connection from various parts of the codebase, it would be possible
that the send channel would be closed and we could still send a message.
This would crash the server.
To fix this, we refactor the code to centralize all sending from the main
hub goroutine. This means we can leverage the connections map to check
if the connection exists or not, and only then send the message.
We also move the cluster calls to cluster.go.
* bring back cluster code inside hub
* Incorporate review comments
* Address review comments
* rename index
* MM-23807: Refactor web_conn
- Unexport some struct fields and constants which are not necessary
to be accessed from outside the package. This will help us moving
the entire websocket handling code to a separate package later.
- Change some empty string checks to check for empty string rather
than doing a len check which is more idiomatic. Both of them compile
to the same code. So it doesn't make a difference performance-wise.
- Remove redundant ToJson calls to get the length.
- Incorporate review comments
- Unexport some more methods
* Fix field name
* Run make app-layers
* Add note on hub check
* MM-22044: Fix panic on web_conn send hello
(*Hub).Start is the central place for sending all web connection
related traffic. However, there was this one call to (*WebConn).Hello
which tried to send a message to a webconn separately.
This was a rare case, but it did occur under stress conditions generated
from a load test.
When the websocket send SEND_QUEUE_SIZE would get filled up and we would
attempt to make a broadcast, the non-blocking send would close the Send
channel of the web connection. During that time, if a web connection
would try to perform a broadcast, it would try to send to a closed channel
and cause a panic.
The solution is to bring back the sending of hello into the same goroutine
inside (*Hub).Start so that all state is centralised and we avoid
sending to a closed channel by sending the hello message inside the registering
code itself.
* Adding non-blocking send
* Simplify things
* Remove test
* Bring sendHello back
* Improve code further
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-21481: Fixed several issues from user marshalling
- Fixed the root cause for panic by properly converting
the map to a User struct.
- Added a check for type conversion for extra safety.
- Fixed a somewhat unrelated issue of a pointer to pointer reference.
* Fix tests
* Make WebSocketEvent type immutable
* Update code to use updated immutable WebSocketEvent type
* Export WebSocketEvent fields and mark them as deprecated
* Consistent license message for all the go files
* Fixing the last set of unconsistencies with the license headers
* Addressing PR review comments
* Fixing busy.go and busy_test.go license header
This change is being made to address an issue where the go-i18n
translation library would result in partial-translations when a
given language dictionary was missing a given plural keyword. The
improvement made here leads the translation library to try an
'other' keyword lookup if the first plural keyword fails to have
a value.
This change was not accepted upstream due to concern regarding
changing the behavior, so we are using a fork at this time to
address the issue.
* fix webconn close semantics
Avoid race conditions in WebConn on shutdown by closing channels to guarantee all readers are notified. Wrap this with sync.Once to avoid closing the channel more than once.
* web_hub_test.go
* webhub: fix race condition on shutdown
Ensure that if the webhub shuts down in the process of sending, the caller unblocks given that the webhub will no longer consume incoming events.
* panic if app shutdown takes >30 seconds
* simplify WebConn::Pump channel semantics too
* Moving goroutine pool
* Auto refactor
* Moving plugins.
* Auto refactor
* Moving fields to server
* Auto refactor
* Removing siteurl duplication.
* Moving reset of app fields
* Auto refactor
* Formatting
* Moving niling of Server to after last use
* Fixing unit tests.
* MM-11065: Allow to search and get archived channels from the API
* Fixing more tests
* Add some unit tests
* Add includeDeleted parameter to session permissions check function
* More test fixing
* Adding archive channels list in channels search
* Add restriction for archived channel edition
* Reverting permissions checks modification
* Changed the query parameter to include_deleted
* Enable search archive channels as true by default
* Adding tests for verify search on deleted channels
* Allowing to override archive channels during the imports
* Fixed test
* Search in archive channels from the API must be explicitly requested
* Removing includeDeleted parameter from GetChannelByName and GetChannelByNameForTeam
* Back to ViewArchivedChannels config
* Fixing tests
* Reverting GetChannelByName parameter
* Add include deleted parameter on GetChannel functions in plugins api
* Fixing tests
* Implementing structured logging
* Changes to en.json to allow refactor to run.
* Fixing global logger
* Structured logger initalization.
* Add caller.
* Do some log redirection.
* Auto refactor
* Cleaning up l4g reference and removing dependancy.
* Removing junk.
* Copyright headers.
* Fixing tests
* Revert "Changes to en.json to allow refactor to run."
This reverts commit fd8249e99bcad0231e6ea65cd77c32aae9a54026.
* Fixing some auto refactor strangeness and typo.
* Making keys more human readable.
* Add user.DeepCopy() function
* Add omit admins/non-admins to WS broadcast and use for updating users
* Updates per feedback and adding unit test for ShouldSendEvent
* Add sequence number to websocket connections and events
* Copy pointer instead of pass by value and use int64 over uint64
* Add more logging to missed events
* Add app package and move logic over from api package
* Change app package functions to return errors
* Move non-api tests into app package
* Fix merge
