MM-52642 CLD-6352 MM-54007 Prepare setup for Playwright pipeline (#24647)

* pipeline(playwright): prepare setup for playwright

* feedback address

* clean up readme

* Fix docker-compose command and definition for dashboard

* fix the dashboard

* ignore dashboard when formatting

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Mario Vitale <mvitale1989@hotmail.com>

e2e-tests/README.md

@@ -17,15 +17,34 @@ Instructions, tl;dr: create a local branch with your E2E test changes, then open
 Instructions, detailed:
 1. (optional, undefined variables are set to sane defaults) Create the `.ci/env` file, and populate it with the variables you need out of the following list:
   * The following variables will be passed over to the server container: `MM_LICENSE` (no enterprise features will be available if this is unset), and the exploded `MM_ENV` (a comma-separated list of env var specifications)
+  * `TEST` either `cypress` (default) or `playwright`.
+  * `SERVER` either `self-hosted` (default) or `cloud`.
+  * The `ENABLED_DOCKER_SERVICES` is by default set to `postgres` and `inbucket` for smoke tests purpose, and for lightweight and faster start up time. Depending on the test requirement being worked on, you may want to override as needed, as such:
+    * Cypress full tests require full services with `postgres`, `inbucket`, `minio`, `openldap`, `elasticsearch` and `keycloak`.
+    * Cypress smoke tests require `postgres` and `inbucket` only.
+    * Playwright full tests require `postgres` and `inbucket` only.
   * The following variables will be passed over to the cypress container: `BRANCH`, `BUILD_ID`, `CI_BASE_URL`, `BROWSER`, `AUTOMATION_DASHBOARD_URL` and `AUTOMATION_DASHBOARD_TOKEN`
   * The `SERVER_IMAGE` variable can also be set, if you want to select a custom mattermost-server image. If not specified, the value of the `SERVER_IMAGE_DEFAULT` variable defined in file `.ci/.e2erc` is used.
   * The `TEST_FILTER` variable can also be set, to customize which tests you want cypress to run. If not specified, only the smoke tests will run. Please check the `e2e-tests/cypress/run_tests.js` file for details about its format.
+  * Set `CWS_URL` when spinning up a cloud-like test server that communicates with a test instance of customer web server. 
 2. (optional) `make start-dashboard`: start the automation-dashboard in the background
   * This also sets the `AUTOMATION_DASHBOARD_URL` and `AUTOMATION_DASHBOARD_TOKEN` variables for the cypress container
-  * Note that if you run the dashboard locally, but also specify other `AUTOMATION_DASHBOARD_*` variables in your `.ci/env` file, the latter variables will take precedence
-3. `make`: start and prepare the server, then run the cypress tests
-  * You can track the progress of the run in the `http://localhost:4000/cycles` dashboard, if you launched it locally
-4. `make stop`: tears down the server (and the dashboard, if running), then cleans up the env placeholder files
+  * Note that if you run the dashboard locally, but also specify other `AUTOMATION_DASHBOARD_*` variables in your `.ci/env` file, the latter variables will take precedence.
+  * The dashboard is used for orchestrating specs with parallel test run and is typically used in CI.
+  * Only Cypress is currently using the dashboard; Playwright is not.
+3. Run the following commands to run with cypress:
+  * `TEST=cypress make`: start and prepare the server, then run the cypress tests against self-hosted test server.
+  * `TEST=cypress SERVER=cloud make`: create test customer for cloud, start and prepare the server, run the cypress tests against cloud-like test server, then delete the test customer. When anticipating a licensed test server, make sure the loaded license via `MM_LICENSE` is for cloud.
+  * When anticipating a licensed test server, make sure the loaded license via `MM_LICENSE` does match either for self-hosted or cloud.
+  * `CWS_URL` is required to be set for cloud test.
+  * You can track the progress of the run in the `http://localhost:4000/cycles` dashboard, if you launched it locally.
+4. Run the following commands to run with playwright:
+  * `TEST=playwright make`: start and prepare the server, then run the playwright tests against self-hosted test server.
+  * `TEST=playwright SERVER=cloud `: generate test customer for cloud, start and prepare the server, run the playwright tests against cloud-like test server, then delete the initially generated test customer.
+  * When anticipating a licensed test server, make sure the loaded license via `MM_LICENSE` does match either for self-hosted or cloud.
+  * `CWS_URL` is required to be set for cloud test.
+5. `TEST=server make`: starts local server.
+6. `make stop`: tears down the server (and the dashboard, if running), then cleans up the env placeholder files.
 
 Notes:
 - Setting a variable in `.ci/env` is functionally equivalent to exporting variables in your current shell's environment, before invoking the makefile.
@@ -34,5 +53,8 @@ Notes:
 - If you restart the dashboard at any point, you must also restart the server containers, so that it picks up the new IP of the dashboard from the newly generated `.env.dashboard` file
 - If you started the dashboard locally in the past, but want to point to another dashboard later, you can run `make clean-env-placeholders` to remove references to the local dashboard (you'll likely need to restart the server)
 - If new variables need to be passed to any of the containers:
-  * If their value is fixed (e.g. a static server configuration): these may be simply added to the `.ci/server.override.yml` file, to the appropriate container.
+  * If their value is fixed (e.g. a static server configuration), these may be simply added to the `docker_compose_generator.sh` file, to the appropriate container.
   * If you need to introduce variables that you want to control from `.ci/env`: you need to update the scripts under the `.ci/` dir, and configure them to write the new variables' values over to the appropriate `.env.*` file. In particular, avoid defining variables that depend on other variables within the docker-compose override files: this is to ensure uniformity in their availability, and simplifies the question of what container has access to which variable considerably.
+
+##### For code changes:
+* `make fmt-ci` to format and check yaml files and shell scripts.