It's was including it in the bin folder but when we package we need to include it in the dist/bin so right now the `download_mmctl` script accepts a second parameter to define the path where the binary is going to be stored
* Update to get latest release from S3
* Update mmctl to download to use S3
* Download MMCTL from S3
* Update download_mmctl_release.sh
* Update Makefile
* Update release.mk
* Update the script called, remove call to sub shell and clean up indentation
* Add quotes around THIS_BRANCH var
* Update script to support overriding the OS for packaging
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Summary:
Every make call would ping Github to find out the mmctl version to download. Each check made 4 requests to Github. So every make execution resulted in 4 requests to Github. This leads to frequent rate-limit errors from Github.
In this PR we check for the mmctl version only if mmctl doesn't already exist. We also print a more helpful error message.
Reduce the number the number of requests to Github from 4 to 2.
Ticket Link:
https://mattermost.atlassian.net/browse/MM-28303
* MM-27512: Use an authenticated user to bump up request rate limit
An unauthenticated user can only make 60 requests per hour which means 1
request every minute. This can lead to frequent rate limit errors while
getting the latest release.
We change that to use an authenticated user which is already available
in the CI. This moves us to make 5000 requests per hour.
We also add additional logging in the Makefile targets in case
the command fails again so that it's clear what has happened, and not return
cryptic 404 errors again.
Ideally, we should be able to inspect the output of the curl command, but since
the output value of the entire bash script is fed into the variable, it is a bit
difficult to print debug output.
If this still gives error, then we need to either use a cached artifact somehow
or add additional logging and add a retry logic on top of it.
* fix mistake
Our codebase uses main_test.go files to control tests at a package level.
However, the problem with that is when we use ./... to build the codebase,
those binaries also get built.
This is unnecessary and creates further delays in the pipeline. All we need to build
are only mattermost and platform binaries in CI. The rest are dependant on the developer.
In my dev environment, this reduces the build times from 13 seconds to 4 seconds.
https://mattermost.atlassian.net/browse/MM-27735
* MM-26584: Bump up Go patch version
This contains a patch fix for https://github.com/golang/go/issues/38023
which have been encountered in our load tests and some users.
I have verified on such a stuck instance and the thread dump seem to match
with what is seen on the issue. All threads stuck on runtime.futex, except one,
which is at runtime.osyield.
And of all the times load tests were run which _did not_ show this issue,
it was run with a version greater than 1.14.1, which further points to the
theory that this bug is the culprit.
* Replace docker image versions.
* Update build/README.md
Co-authored-by: Elisabeth Kulzer <elikul@elikul.de>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Configurable dev environment
* Add a bit of documentation
* fixing gofmt
* A bit more doc
* Using variable
* Adding license header
* Moving LDAP_DATA variable to the default-config.mk file
* Adding another docker-compose for the makefile to not brake anybody workflow
* Moving dejavu to the config
* Fixing docker-compose.makefile.yaml for dejavu
* Adding keycloak support to the dev environment
* Address PR review comments
* Removing minio from default docker images
* Changing the default version of mysql to the oldest supported (5.6)
* Change the restart option to no for the dev environment
* Fixing restart option
* Reverting unneded changes
* Restoring 5.7 to check if test passes
* Going back to 5.6 mysql image
* Fixing tests on mysql 5.6
* Skipping flaky test
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
We should not use such a long `HEALTHCHECK` `--interval` as first task check if happening after given time, and before healthcheck returns `0` – **task won't be available** to the Docker networks – meaning that Mattermost won't work until then even if it deployed correctly!
And in Docker envs it happens often – that DB is available all the time, and just MM app is being updated. In this case even if we are healthy since 1s – we need to wait 5min until it goes online.
* Jenkins: Make mysql and postgres tests run in parallel
* Trying again
* Start docker containers individually for each stage
This is because the parallel block does not allow
a common step to be specified.
* Giving a random name to each docker-compose
* Fixing order of the param
* Giving unique project names for tests
* Setting the right datasource and drivername
* Trying to use different copies of the src directory for test the databas3
* moving workspace copy to the beginning of postgres tests
* Fixing some missed out patches
* Added missing LDAP population command
* Upload all test results
And set allowEmptyResults to false.
* Pruning remaining docker networks
* Remove unneeded settings
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Jesús Espino <jespinog@gmail.com>
* Running tests in mysql and postgres
* Fixing ci
* Fixing ci
* Fixing ci
* Fixing ci
* Fixing ci
* Fixing ci
* Fixing ci
* Fixing ci
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* Updating build and release process to include externally built mmctl
* Missed last commit before reset
* rename script to align with purpose
* Moving get release script to scripts dir
* Streamlined script to find the correct latest version of an externally built binary from a GitHub Release
* Updated variable names to match
* Updating vars to match
* Updating parameters to named variables for clarity
* Update scripts/get_latest_release.sh
Co-Authored-By: Elisabeth Kulzer <elikul@elikul.de>
Co-authored-by: Elisabeth Kulzer <elikul@elikul.de>
* Run all tests fasts in postgres
* Moving postgres config to a file
* Addressing PR review comments
* Testing against mysql in the CI to keep things less changed
* Revert "Testing against mysql in the CI to keep things less changed"
This reverts commit fc940c111db269a821c54e005d76df87cb70e031.
* Fixing a test broken in postgres but working in mysql
* Fixing some tests
* Fixing some config tests with postgres
* Fixing the rest of config tests
* Fixing govet error
* Fixing search tests for postgres
* Updating build and release process to include externally built mmctl
* Missed last commit before reset
* rename script to align with purpose
* Moving get release script to scripts dir
* MM-21769 - Validate plugin bundle and signature files when packaging MM
* Renamed public key, using gpg exit code to verify success
* Update Makefile
Co-Authored-By: Jesse Hallam <jesse.hallam@gmail.com>
* Pulling platform specific prepackaged plugins
* Verify platform specific plugin binary exist when building each ARCH
Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
Go 1.13 introduced this new flag which removes all absolute file system paths
from the binary and just keeps the module path / GOPATH. To debug binary
crashes or stack traces, we only need the path to the code and not the full
file path. Hence this is a quick way to reduce the binary size without any information
loss.
Shaves off around 750KB from the server binary.
* Add the old fmt and vet targets to legacy.mk
This is to ease the transition until a time
when everybody has moved to using golangci-lint.
* Just run golangci-lint for make vet
