Mono repo -> Master (#22553)

Combines the following repositories into one:

https://github.com/mattermost/mattermost-server
https://github.com/mattermost/mattermost-webapp
https://github.com/mattermost/focalboard
https://github.com/mattermost/mattermost-plugin-playbooks

server/channels/product/README.md

@@ -0,0 +1,114 @@
+# Product
+
+Package product defines the interfaces provided in the multi-product architecture framework. The service interfaces are designed to be a drop in replacement for services defined in the https://github.com/mattermost/mattermost-plugin-api project. Due to limitations such as the use of https://github.com/mattermost/mattermost-server/blob/master/plugin/api.go emerged this new API. Our hope is to use a single API definition or maybe even more interesting solutions like using the app.AppIFace instead (temporarily).
+
+## Multi-product architecture framework
+
+The main goal of multi-product architecture effort is to divide the prominent “app” package into sub packages so that we can maintain the complexity and lay the groundwork for future scaling opportunities. And the framework is the implementation of this idea. Currently the framework is very early to be stable and it's going to be evolve in time once we start using it.
+
+### How does the framework work?
+
+A product should conform to the following interface:
+
+```Go
+type Product interface {
+	Start() error
+	Stop() error
+}
+```
+
+The `app.Server` will take care of starting and stopping products. The product shall register itself via a function called `RegisterProduct` provided by `github.com/mattermost/mattermost-server/v6/server/app` package. To register a product,
+a product initializer is required. The signature of a product initializer is defined as following:
+
+```Go
+type app.ProductManifest struct {
+	Initializer  func(*app.Server, map[app.ServiceKey]interface{}) (app.Product, error)
+	Dependencies map[app.ServiceKey]struct{}
+}
+```
+
+Note that adding dependencies is crucial to let product framework sort product initialization. For example Channels product provides the `product.PostService` implementation therefore it should be initialized before the Boards product since it requires the PostService. An example registration could be depicted as following:
+
+```Go
+func init() {
+	app.RegisterProduct("focalboard", app.ProductManifest{
+		Initializer:  NewBoards,
+		Dependencies: map[app.ServiceKey]struct{}{
+            app.PostKey:        {},
+			app.PermissionsKey: {},
+			app.UserKey:        {},
+			...
+        },
+	})
+}
+```
+
+### Adding services to the framework
+
+A product can provide services to the framework. In fact, `Channels` product provides many services by itself, so it will only need to register the service to `services` map provided by the product initializer. An example of registering a service to the "registry" is shown below:
+
+```Go
+func NewChannels(*app.Server, map[app.ServiceKey]interface{}) (app.Product, error){
+	...
+	services[app.PostKey] = &postService{
+		...
+	}
+	...
+}
+```
+
+To improve the developer experience, you should also add the service interface to the [api definition](api.go) so that a consumer of the service can explore the methods available to them. Another good practice would be to add the servie key to the [server.go](../app/server.go) file.
+
+### How does a product get initialized?
+
+The overall server initialization starts with essential components such as the store, config etc. Right after that we start to initialize the services which are either a standalone service such as the `FileStore` and `UserService` or some services which are eventually wrappers to the server struct itself such as `ClusterService` and `LicenseService`. And the initial service map is created after these stages.
+
+```Go
+func NewServer(options ...Option) (*Server, error) {
+	...
+	s := &Server{}
+	...
+	serviceMap := map[ServiceKey]interface{}{
+		...
+	}
+
+	if err := s.initializeProducts(products, serviceMap); err != nil {
+		return nil, errors.Wrap(err, "failed to initialize products")
+	}
+	...
+}
+```
+
+And the product initialization is figured out by a trial and error fashion hence it is done by a maximum possible trials of initialization attempts. The order is not determined elsewhere therefore we do a on the fly sorting here. Which means the initialization order will be resolved during the loop.  We have dependencies defined in the product manifest defined above. During the initialization we check if the serviceMap has all the dependencies registered. If not, we continue to the try initialize other products and register their services if they have any.
+
+### How to add a product to the mattermost-server?
+
+We don't need to define a product dependency in the `go.mod` file, we can leverage the [module workspaces](https://go.dev/ref/mod#workspaces) here.  You can get more info about how we use it [here](https://docs.google.com/document/d/1Uwg_dTSNR9mx9ZDx-7osjlD4n3w13cnG6Kpz3ZGXzsM). We create another file such as `go.work`, and add the dependency there as following:
+
+```
+go 1.18
+use ./
+use ../sample-product
+```
+
+This tells the compiler to include `sample-product` to be compiled with the mattermost-server. And in order to trigger `init()` function of a product we add an empty import to a file as following:
+
+```Go
+package imports
+
+import (
+	...
+	// Product Imports
+	_ "github.com/mattermost/focalboard/product"
+)
+```
+
+### Frequently asked questions
+
+#### Can a product use app.App instead of services?
+
+Theoretically yes, but you shouldn't. The reason is we want to figure out the common entry points and use cases for the services so that we can divide the App into meaningful and functional sub services. The current service interfaces are great example of how we want to use the services among the products.
+
+#### How to handle circular dependency of two products?
+
+We are not expecting this is a requirement for the initial phase, once we complete the first product migration we can think start thinking about this. The first attempt would be to increase the granularity of the initialization phase by adopting service initialization resolution. So that a service can be initialized even a product initialization starts.

server/channels/product/api.go

@@ -0,0 +1,264 @@
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
+// See LICENSE.txt for license information.
+
+package product
+
+import (
+	"database/sql"
+
+	"github.com/gorilla/mux"
+
+	"github.com/mattermost/mattermost-server/v6/model"
+	"github.com/mattermost/mattermost-server/v6/server/channels/app/request"
+	"github.com/mattermost/mattermost-server/v6/server/platform/shared/filestore"
+	"github.com/mattermost/mattermost-server/v6/server/platform/shared/mlog"
+
+	fb_model "github.com/mattermost/mattermost-server/v6/server/boards/model"
+)
+
+// RouterService enables registering the product router to the server. After registering the
+// router, the ServeHTTP hook which was being used in plugin mode is not required anymore.
+// For now, the service implementation is provided by Channels therefore the consumer products
+// should add this service key to their dependencies map in the app.ProductManifest.
+//
+// The service shall be registered via app.RouterKey service key.
+type RouterService interface {
+	RegisterRouter(productID string, sub *mux.Router)
+}
+
+// PostService provides posts related utilities.  For now, the service implementation
+// is provided by Channels therefore the consumer products should add this service key to
+// their dependencies map in the app.ProductManifest.
+//
+// The service shall be registered via app.PostKey service key.
+type PostService interface {
+	CreatePost(context *request.Context, post *model.Post) (*model.Post, *model.AppError)
+	GetPostsByIds(postIDs []string) ([]*model.Post, int64, *model.AppError)
+	SendEphemeralPost(ctx *request.Context, userID string, post *model.Post) *model.Post
+	GetPost(postID string) (*model.Post, *model.AppError)
+	DeletePost(ctx *request.Context, postID, productID string) (*model.Post, *model.AppError)
+	UpdatePost(c *request.Context, post *model.Post, safeUpdate bool) (*model.Post, *model.AppError)
+}
+
+// PermissionService provides permissions related utilities. For now, the service implementation
+// is provided by Channels therefore the consumer products should add this service key to their
+// dependencies map in the app.ProductManifest.
+//
+// The service shall be registered via app.PermissionKey service key.
+type PermissionService interface {
+	HasPermissionTo(userID string, permission *model.Permission) bool
+	HasPermissionToTeam(userID, teamID string, permission *model.Permission) bool
+	HasPermissionToChannel(askingUserID string, channelID string, permission *model.Permission) bool
+	RolesGrantPermission(roleNames []string, permissionID string) bool
+}
+
+// ClusterService enables to publish cluster events. In addition to that, It's being used for
+// mattermost-plugin-api Mutex API with the SetPluginKeyWithOptions method.
+//
+// The service shall be registered via app.ClusterKey key.
+type ClusterService interface {
+	PublishPluginClusterEvent(productID string, ev model.PluginClusterEvent, opts model.PluginClusterEventSendOptions) error
+	PublishWebSocketEvent(productID string, event string, payload map[string]any, broadcast *model.WebsocketBroadcast)
+}
+
+// ChannelService provides channel related API  The service implementation is provided by
+// Channels product therefore the consumer products should add this service key to their
+// dependencies map in the app.ProductManifest.
+//
+// The service shall be registered via app.ChannelKey service key.
+type ChannelService interface {
+	GetDirectChannel(userID1, userID2 string) (*model.Channel, *model.AppError)
+	GetDirectChannelOrCreate(userID1, userID2 string) (*model.Channel, *model.AppError)
+	GetChannelByID(channelID string) (*model.Channel, *model.AppError)
+	GetChannelMember(channelID string, userID string) (*model.ChannelMember, *model.AppError)
+	GetChannelsForTeamForUser(teamID string, userID string, opts *model.ChannelSearchOpts) (model.ChannelList, *model.AppError)
+	GetChannelSidebarCategories(userID, teamID string) (*model.OrderedSidebarCategories, *model.AppError)
+	GetChannelMembers(channelID string, page, perPage int) (model.ChannelMembers, *model.AppError)
+	CreateChannelSidebarCategory(userID, teamID string, newCategory *model.SidebarCategoryWithChannels) (*model.SidebarCategoryWithChannels, *model.AppError)
+	UpdateChannelSidebarCategories(userID, teamID string, categories []*model.SidebarCategoryWithChannels) ([]*model.SidebarCategoryWithChannels, *model.AppError)
+	CreateChannel(channel *model.Channel) (*model.Channel, *model.AppError)
+	AddUserToChannel(channelID, userID, asUserID string) (*model.ChannelMember, *model.AppError)
+	UpdateChannelMemberRoles(channelID, userID, newRoles string) (*model.ChannelMember, *model.AppError)
+	DeleteChannelMember(channelID, userID string) *model.AppError
+	AddChannelMember(channelID, userID string) (*model.ChannelMember, *model.AppError)
+}
+
+// LicenseService provides license related utilities.
+//
+// The service shall be registered via app.LicenseKey service key.
+type LicenseService interface {
+	GetLicense() *model.License
+	RequestTrialLicense(requesterID string, users int, termsAccepted bool, receiveEmailsAccepted bool) *model.AppError
+}
+
+// UserService provides user related utilities. Initially this was thought to be app/users.UserService
+// but it's replaced by app.App temporarily. The reason is; UserService is a standalone tool whereas the
+// existing plugin API was using channels related app functionalities as well. We shall improve the UserService
+// to meet emerging requirements.
+//
+// The service shall be registered via app.UserKey service key.
+type UserService interface {
+	GetUser(userID string) (*model.User, *model.AppError)
+	UpdateUser(c request.CTX, user *model.User, sendNotifications bool) (*model.User, *model.AppError)
+	GetUserByEmail(email string) (*model.User, *model.AppError)
+	GetUserByUsername(username string) (*model.User, *model.AppError)
+	GetUsersFromProfiles(options *model.UserGetOptions) ([]*model.User, *model.AppError)
+}
+
+// TeamService provides team related utilities.
+//
+// The service shall be registered via app.TeamKey service key.
+type TeamService interface {
+	GetMember(teamID, userID string) (*model.TeamMember, *model.AppError)
+	CreateMember(ctx *request.Context, teamID, userID string) (*model.TeamMember, *model.AppError)
+	GetGroup(groupId string) (*model.Group, *model.AppError)
+	GetTeam(teamID string) (*model.Team, *model.AppError)
+	GetGroupMemberUsers(groupID string, page, perPage int) ([]*model.User, *model.AppError)
+}
+
+// BotService is just a copy implementation of mattermost-plugin-api EnsureBot method.
+//
+// The service shall be registered via app.BotKey service key.
+type BotService interface {
+	EnsureBot(ctx *request.Context, productID string, bot *model.Bot) (string, error)
+}
+
+// ConfigService shall be registered via app.ConfigKey service key.
+type ConfigService interface {
+	Config() *model.Config
+	AddConfigListener(listener func(*model.Config, *model.Config)) string
+	RemoveConfigListener(id string)
+	UpdateConfig(f func(*model.Config))
+	SaveConfig(newCfg *model.Config, sendConfigChangeClusterMessage bool) (*model.Config, *model.Config, *model.AppError)
+}
+
+// HooksService is the API for adding exiting plugin hooks to the server so that they can be called as
+// they were. This Service is required to be accessed after the channels product initialized.
+//
+// The service shall be registered via app.HooksKey service key.
+type HooksService interface {
+	// RegisterHook checks whether if the 'hooks' implements any method of plugin.Hooks methods. Rather than
+	// using the whole plugin.Hooks interface with its 20+ methods, a product can implement any exiting method
+	// of plugin.Hooks w/o requiring to declare which method they implemented or not. This is going to be
+	// checked on runtime. We have individual interfaces for each method declared in plugin.Hooks interface.
+	// Hence, while registering a product, the service will check if the product implements any of these individual
+	// interfaces. If so, a map of hook IDs that are implemented will be used to call the hooks. The method will
+	// return an error in case if there is an incorrect implementation of the any of the individual interface in runtime.
+	// Consider checking plugin.Hooks for the reference.
+	// Following methods are not allowed to be implemented in the product:
+	//  - plugin.Hooks.OnActivate
+	//  - plugin.Hooks.OnDeactivate
+	//  - plugin.Hooks.Implemented
+	//  - plugin.Hooks.ServeHTTP
+	RegisterHooks(productID string, hooks any) error
+}
+
+// FilestoreService is the API for accessing the file store.
+//
+// The service shall be registered via app.FilestoreKey service key.
+type FilestoreService interface {
+	filestore.FileBackend
+}
+
+// FileInfoStoreService is the API for accessing the file info store.
+//
+// The service shall be registered via app.FileInfoStoreKey service key.
+type FileInfoStoreService interface {
+	GetFileInfo(fileID string) (*model.FileInfo, *model.AppError)
+}
+
+// CloudService is the API for accessing the cloud service APIs.
+//
+// The service shall be registered via app.CloudKey service key.
+type CloudService interface {
+	GetCloudLimits() (*model.ProductLimits, error)
+}
+
+// KVStoreService is the API for accessing the KVStore service APIs.
+//
+// The service shall be registered via app.KVStoreKey service key.
+type KVStoreService interface {
+	SetPluginKeyWithOptions(pluginID string, key string, value []byte, options model.PluginKVSetOptions) (bool, *model.AppError)
+	KVGet(productID, key string) ([]byte, *model.AppError)
+	KVDelete(productID, key string) *model.AppError
+	KVList(productID string, page, perPage int) ([]string, *model.AppError)
+}
+
+// LogService is the API for accessing the log service APIs.
+//
+// The service shall be registered via app.LogKey service key.
+type LogService interface {
+	mlog.LoggerIFace
+}
+
+// StoreService is the API for accessing the Store service APIs.
+//
+// The service shall be registered via app.StoreKey service key.
+type StoreService interface {
+	GetMasterDB() *sql.DB
+}
+
+// SystemService is the API for accessing the System service APIs.
+//
+// The service shall be registered via app.SystemKey service key.
+type SystemService interface {
+	GetDiagnosticId() string
+}
+
+// PreferencesService is the API for accessing the Preferences service APIs.
+//
+// The service shall be registered via app.PreferencesKey service key.
+type PreferencesService interface {
+	GetPreferencesForUser(userID string) (model.Preferences, *model.AppError)
+	UpdatePreferencesForUser(userID string, preferences model.Preferences) *model.AppError
+	DeletePreferencesForUser(userID string, preferences model.Preferences) *model.AppError
+}
+
+// BoardsService is the API for accessing Boards service APIs.
+//
+// The service shall be registered via app.BoardsKey service key.
+type BoardsService interface {
+	GetTemplates(teamID string, userID string) ([]*fb_model.Board, error)
+	GetBoard(boardID string) (*fb_model.Board, error)
+	CreateBoard(board *fb_model.Board, userID string, addmember bool) (*fb_model.Board, error)
+	PatchBoard(boardPatch *fb_model.BoardPatch, boardID string, userID string) (*fb_model.Board, error)
+	DeleteBoard(boardID string, userID string) error
+	SearchBoards(searchTerm string, searchField fb_model.BoardSearchField, userID string, includePublicBoards bool) ([]*fb_model.Board, error)
+	LinkBoardToChannel(boardID string, channelID string, userID string) (*fb_model.Board, error)
+	GetCards(boardID string) ([]*fb_model.Card, error)
+	GetCard(cardID string) (*fb_model.Card, error)
+	CreateCard(card *fb_model.Card, boardID string, userID string) (*fb_model.Card, error)
+	PatchCard(cardPatch *fb_model.CardPatch, cardID string, userID string) (*fb_model.Card, error)
+	DeleteCard(cardID string, userID string) error
+	HasPermissionToBoard(userID, boardID string, permission *model.Permission) bool
+	DuplicateBoard(boardID string, userID string, toTeam string, asTemplate bool) (*fb_model.BoardsAndBlocks, []*fb_model.BoardMember, error)
+}
+
+// SessionService is the API for accessing the session.
+//
+// The service shall be registered via app.SessionKey service key.
+type SessionService interface {
+	GetSessionById(sessionID string) (*model.Session, *model.AppError)
+}
+
+// FrontendService is the API for interacting with front end.
+//
+// The service shall be registered via app.FrontendKey service key.
+type FrontendService interface {
+	OpenInteractiveDialog(dialog model.OpenDialogRequest) *model.AppError
+}
+
+// CommandService is the API for interacting with front end.
+//
+// The service shall be registered via app.CommandKey service key.
+type CommandService interface {
+	ExecuteCommand(c request.CTX, args *model.CommandArgs) (*model.CommandResponse, *model.AppError)
+	RegisterProductCommand(productID string, command *model.Command) error
+}
+
+// ThreadsService is the API for interacting with threads anywhere.
+//
+// The service shall be registered via app.ThreadsKey service key.
+type ThreadsService interface {
+	RegisterCollectionAndTopic(productID string, collectionType, topicType string) error
+}

server/channels/product/doc.go

@@ -0,0 +1,10 @@
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
+// See LICENSE.txt for license information.
+
+// Package product defines the interfaces provided in the multi-product architecture
+// framework. The service interfaces are designed to be a drop in replacement for services
+// defined in the https://github.com/mattermost/mattermost-plugin-api project. Due to limitations
+// such as the use of https://github.com/mattermost/mattermost-server/blob/master/plugin/api.go
+// emerged this new API. Our hope is to use a single API definition or maybe even more interesting
+// solutions like using the app.AppIFace instead.
+package product

server/channels/product/hooks.go

@@ -0,0 +1,79 @@
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
+// See LICENSE.txt for license information.
+
+package product
+
+import (
+	"sync"
+	"time"
+
+	"github.com/mattermost/mattermost-server/v6/plugin"
+	"github.com/mattermost/mattermost-server/v6/server/channels/einterfaces"
+)
+
+type HooksManager struct {
+	registeredProducts sync.Map
+	metrics            einterfaces.MetricsInterface
+}
+
+func NewHooksManager(metrics einterfaces.MetricsInterface) *HooksManager {
+	return &HooksManager{
+		metrics: metrics,
+	}
+}
+
+func (m *HooksManager) AddProduct(productID string, hooks any) error {
+	prod, err := plugin.NewAdapter(hooks)
+	if err != nil {
+		return err
+	}
+
+	rp := &plugin.RegisteredProduct{
+		ProductID: productID,
+		Adapter:   prod,
+	}
+
+	m.registeredProducts.Store(productID, rp)
+
+	return nil
+}
+
+func (m *HooksManager) RemoveProduct(productID string) {
+	m.registeredProducts.Delete(productID)
+}
+
+func (m *HooksManager) RunMultiHook(hookRunnerFunc func(hooks plugin.Hooks) bool, hookId int) {
+	startTime := time.Now()
+
+	m.registeredProducts.Range(func(key, value any) bool {
+		rp := value.(*plugin.RegisteredProduct)
+
+		if !rp.Implements(hookId) {
+			return true
+		}
+
+		hookStartTime := time.Now()
+		result := hookRunnerFunc(rp.Adapter)
+
+		if m.metrics != nil {
+			elapsedTime := float64(time.Since(hookStartTime)) / float64(time.Second)
+			m.metrics.ObservePluginMultiHookIterationDuration(rp.ProductID, elapsedTime)
+		}
+
+		return result
+	})
+
+	if m.metrics != nil {
+		elapsedTime := float64(time.Since(startTime)) / float64(time.Second)
+		m.metrics.ObservePluginMultiHookDuration(elapsedTime)
+	}
+}
+
+func (m *HooksManager) HooksForProduct(id string) plugin.Hooks {
+	if value, ok := m.registeredProducts.Load(id); ok {
+		rp := value.(*plugin.RegisteredProduct)
+		return rp.Adapter
+	}
+
+	return nil
+}

server/channels/product/product.go

@@ -0,0 +1,24 @@
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
+// See LICENSE.txt for license information.
+
+package product
+
+type Product interface {
+	Start() error
+	Stop() error
+}
+
+type Manifest struct {
+	Initializer  func(map[ServiceKey]any) (Product, error)
+	Dependencies map[ServiceKey]struct{}
+}
+
+var products = make(map[string]Manifest)
+
+func RegisterProduct(name string, m Manifest) {
+	products[name] = m
+}
+
+func GetProducts() map[string]Manifest {
+	return products
+}

server/channels/product/service.go

@@ -0,0 +1,33 @@
+// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
+// See LICENSE.txt for license information.
+
+package product
+
+type ServiceKey string
+
+const (
+	ChannelKey       ServiceKey = "channel"
+	ConfigKey        ServiceKey = "config"
+	LicenseKey       ServiceKey = "license"
+	FilestoreKey     ServiceKey = "filestore"
+	FileInfoStoreKey ServiceKey = "fileinfostore"
+	ClusterKey       ServiceKey = "cluster"
+	CloudKey         ServiceKey = "cloud"
+	PostKey          ServiceKey = "post"
+	TeamKey          ServiceKey = "team"
+	UserKey          ServiceKey = "user"
+	PermissionsKey   ServiceKey = "permissions"
+	RouterKey        ServiceKey = "router"
+	BotKey           ServiceKey = "bot"
+	LogKey           ServiceKey = "log"
+	HooksKey         ServiceKey = "hooks"
+	KVStoreKey       ServiceKey = "kvstore"
+	StoreKey         ServiceKey = "storekey"
+	SystemKey        ServiceKey = "systemkey"
+	PreferencesKey   ServiceKey = "preferenceskey"
+	BoardsKey        ServiceKey = "boards"
+	SessionKey       ServiceKey = "sessionkey"
+	FrontendKey      ServiceKey = "frontendkey"
+	CommandKey       ServiceKey = "commandkey"
+	ThreadsKey       ServiceKey = "threadskey"
+)