* Create scheduled post api (#27920)

* Added migration files for Postgres

* Added migrations for MySQL

* Added store method

* Added API and store tests

* Renamed migration after syncing with master

* Added app layer tests

* API is ready

* API is ready

* API is ready

* Renamed migration after syncing with master

* Updated migration list

* Fixed retry layer tests

* Allowed posts with empty messages

* Review fixes

* Reverted an incorrect change

* Renamed migration and fixed ID assignment

* CI

* Send post button changes (#28019)

* added Split button

* WIP

* Added core menu options

* WIP

* WIP

* WIP

* Handled displaying error in creating scheduled post

* lint fixes

* webapp i18n fix

* Review fixes

* Fixed a webapp test

* A few more fixes

* Removed a duplicate comment

* Scheduled post job (#28088)

* Added the job function

* Added query for fetching scheduled posts for pricessing

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* Reafactoring of scheduled post job

* Lint fixes

* Updated i18n files

* FInishing touches

* Added tests for GetScheduledPosts

* Added tests for PermanentlyDeleteScheduledPosts

* Updated all layer

* Some changes as discussed with team

* Added tests for UpdatedScheduledPost

* Code review refactoring

* Added job test

* MM-60120 - Custom time selection (#28120)

* Added a common date time picker modal and used it for post reminder

* Added a common date time picker modal and used it for post reminderggp

* Added modal for custom schedule time and fixed TZ issue

* WIP

* Removed event from useSubmit hook

* Removed event from useSubmit hook

* Added timezone handling

* fixed type error

* Updated i18n strings

* Minor cleanup

* updated snapshots

* review fixes

* Handled event

* Supported for having a DM thread open in RHS while in a regular channel

* Review fixes

* MM-60136 - Scheduled messages tab (#28133)

* WIP

* WIP

* Created Tabs and Tab wrapper with added styling

* Added API to get scheduled posts

* WIP

* Displated scheduled post count

* i18n fix

* Added tests

* Handled asetting active tab absed on URL:

* Reverted unintended change

* Added API to client ad OpenAPI specs

* Renamed file

* Adding fileinfo to schedule posts

* Partial review fixes

* Made get scheduled post API return posts by teamID

* review fixes

* Moved scheduled post redux code to MM-redux package

* Usedd selector factory

* WIP:

* WIP:

* Lint fix

* Fixed an incorrect openapi spec file

* Removed redundent permission check

* Clreaed scheduled post data on logout

* Removed unused i18n string:

* lint fix

* Render scheduled posts (#28208)

* WIP

* WIP

* Created Tabs and Tab wrapper with added styling

* Added API to get scheduled posts

* WIP

* Displated scheduled post count

* i18n fix

* Added tests

* Handled asetting active tab absed on URL:

* Reverted unintended change

* Added API to client ad OpenAPI specs

* Renamed file

* Created common component for draft list item

* WIP

* WIP

* Adding fileinfo to schedule posts

* Basic rendering

* Added count badge to tabs

* WIP

* Made the Drafts LHS iteam appear if no drafts exist but scheduled posts do

* Fixed icon size

* Partial review fixes

* Made get scheduled post API return posts by teamID

* Handled initial vs team switch load

* Displayed scheduled date in panel header

* Added error message and error indiocator

* WIP

* review fixes

* WIP Adding error reason tag

* Added error codes

* Moved scheduled post redux code to MM-redux package

* Usedd selector factory

* WIP:

* WIP:

* Lint fix

* Fixed an incorrect openapi spec file

* Removed redundent permission check

* Clreaed scheduled post data on logout

* Removed unused i18n string:

* lint fix

* Opened rescheduling modal

* Updated graphic for empty state of schduled post list

* Added delete scheduled post option and modal

* Badge and timezone fix

* WIP:

* Added send now confirmation modal

* lint

* Webapp i18n fix

* Fixed webapp test

* Fixed a bug where DM/GM scheduled posts weren't immideatly showing up in UI

* Minor fixes

* WIP

* Review fixes

* Review fixes

* Optimisations

* Fixed reducer name

* Moment optimizatin

* Updated route check

* MM-60144 - added API to update a scheduled post (#28248)

* WIP

* Added api and ap layer for update scheduled post ̛̦̄

* Added API to OpenAI specs, Go client and TS client

* removed permissio check

* Added tests

* Fixed tests

* Added PreUpdate method on scheduled post model

* MM-60131 - Reschedule post integration (#28281)

* Handled rescheduling post in webapp

* Added error handling

* MM-60146 - Delete scheduled post api (#28265)

* WIP

* Added api and ap layer for update scheduled post ̛̦̄

* Added API to OpenAI specs, Go client and TS client

* removed permissio check

* Added tests

* Fixed tests

* Added PreUpdate method on scheduled post model

* Added delete scheduled post API

* Added API to Go client and OpenAPI specs

* Added API to TS client

* Added tests

* CI

* Rmeoved two incorrect code comments

* MM-60653 - Integrated delete scheduled post API (#28296)

* Integrated delete scheduled apost API

* Lint fix

* Review fixes

* Excluded draft checks from scheduled posts (#28370)

* Excluded draft checks from scheduled posts

* Added a removed todo

* MM-60125 - Scheduled post channel indicator (#28320)

* Integrated delete scheduled apost API

* Lint fix

* Added state for storing scheduled posts by channel ID

* Refactored redux store to store scheudled posts by ID, thens tore IDs everywhere

* Refactored redux store to store scheudled posts by ID, thens tore IDs everywhere

* WIP

* Added scheduled post indiocator

* Handled single and multiple scheudled posts

* Review fixes

* Fixed styling and handled center channel, RHS and threads view

* Lint fix

* i18n fix

* Fixed a cycling dependency

* Lint fix

* Added some more comments

* Updated styling

* Review fixes

* Added common component for remote user time and scheduled post indicator

* Updated scheduled post count

* Minor change

* Moved CSS code around

* Fixed a bug where files in scheduled post didn't show up until refresh (#28359)

---------

Co-authored-by: Daniel Espino García <larkox@gmail.com>

* Scheduled post config (#28485)

* Added config

* Added config on server and webapp side

* Added config check in server and webapp

* Added license check

* Added license check

* Added placeholder help text

* Added license check to job

* Fixed job test

* Review fixes

* Updated English text

* Review fixes

* MM-60118 - Added index on ScheduledPosts table (#28579)

* Added index

* Updated indexes

* Scheduled posts misc fixes (#28625)

* Added detailed logging for scheduled post job

* Limited scheduled posts processing to 24 hours

* Marked old scheduled posts as unable to send

* Added t5ests

* converted some logs to trace level

* Fixed a bug causing error message to show up on deleting a scheduled post in a deleted thread (#28630)

* Fixed scheduled posts link in RHS (#28659)

* Fixed scheduled posts link in RHS

* Review fixes

* Fix permission name in scheduled posts by team (#28580)

* Fix permission name

* fix wording

---------

Co-authored-by: Mattermost Build <build@mattermost.com>

* FIxed width of generic modal header to fix browser channel modal (#28639)

* Only consider error-free scheduled posts for indicator in channel and RHS (#28683)

* Show only errro free scheudled posts in post box indicator

* Fixed a bug to handle no scheduled posts

* Fixed draft and scheudled post UI in mobile view (#28680)

* MM-60873 and MM-60872 - Fixed a bug with updating scheduled posts (#28656)

* Fixed a bug with updating scheduled posts

* Better selectors

* MOved shceuled post message length validation to app layer

* MM-60732 - Scheduled posts channel link now takes you to the first scheduled post in channel/thread in list (#28768)

* Ordered scheudle dposts by schgeudled at nad create at

* Ordered in client

* Added scroll to target

* Removed classname prop

* Fixed tests

* Added doc

* Import fix

* MM-60961 - Fixed a bug where API used incoming create at date for scheduled post (#28703)

* Fixed a bug where API used incoming create at date for scheduled post

* Stopped sending created at value for scheduled post

* MM-60785 - Fixed a bug where scheduled posts of channel we are no longer member of didn't show up (#28637)

* Fixed a bug where scheduled posts of channel we are no longer member of didn't show up

* Added a comment

* CI

* Used data loader to optimise laoding missing channels

* Minor refactoring

* MM-60963 - Added common checks for post and scheduled posts (#28713)

* Added commen checks for post and scheuled posts

* Sanitised scheduled posts

* Fixed tests

* Splitted post checks into app and context functions

* Added checks on scheduiled posts job as well:

* i18n fix

* Fixed a test

* Renamed a func

* removed duplicate check

* Scheduled posts UI fixes (#28828)

* Fixed send button and time picker borders

* Fixed center alignment of time picker

* Removed on for today and tomorrow

* Lint fix

* Date time modal hover state fix

* Badge fix

* Fixed a mnerge issue

* Scheduled Post send now and add schedule on draft (#28851)

* Added send now option on scheduled posts

* Minor refactoring

* WIP

* WIP

* WIP

* Lint fix

* i18n fix

* Snapshot update

* Review fixes

* Scheduled post inline editing (#28893)

* Added send now option on scheduled posts

* Minor refactoring

* WIP

* WIP

* WIP

* Lint fix

* i18n fix

* Snapshot update

* Displayed editing component in scheduled post

* Added handling for updating scheduled post

* Handle events

* Fixed escape key issue in scheudled post editing

* Fixes

* Displayed error message for editing error

* Don't show mention warning

* Handled dev mode (#28918)

* MInor fixes

* client fix

* Fixes

* CI

* Removed dev mode behaviour temperorily (#29008)

---------

Co-authored-by: Daniel Espino García <larkox@gmail.com>
Co-authored-by: Eva Sarafianou <eva.sarafianou@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
Этот коммит содержится в:
Harshil Sharma
2024-11-04 11:39:35 +05:30
коммит произвёл GitHub
родитель aaf9234c8e
Коммит e281b3f37e
135 изменённых файлов: 7792 добавлений и 684 удалений

Просмотреть файл

@@ -0,0 +1,289 @@
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
// See LICENSE.txt for license information.
package sqlstore
import (
"strings"
"sync"
"github.com/mattermost/mattermost/server/public/model"
"github.com/mattermost/mattermost/server/public/shared/mlog"
sq "github.com/mattermost/squirrel"
"github.com/pkg/errors"
)
type SqlScheduledPostStore struct {
*SqlStore
maxMessageSizeOnce sync.Once
maxMessageSizeCached int
}
func newScheduledPostStore(sqlStore *SqlStore) *SqlScheduledPostStore {
return &SqlScheduledPostStore{
SqlStore: sqlStore,
maxMessageSizeCached: model.PostMessageMaxRunesV2,
}
}
func (s *SqlScheduledPostStore) columns(prefix string) []string {
if prefix != "" && !strings.HasSuffix(prefix, ".") {
prefix = prefix + "."
}
return []string{
prefix + "Id",
prefix + "CreateAt",
prefix + "UpdateAt",
prefix + "UserId",
prefix + "ChannelId",
prefix + "RootId",
prefix + "Message",
prefix + "Props",
prefix + "FileIds",
prefix + "Priority",
prefix + "ScheduledAt",
prefix + "ProcessedAt",
prefix + "ErrorCode",
}
}
func (s *SqlScheduledPostStore) scheduledPostToSlice(scheduledPost *model.ScheduledPost) []interface{} {
return []interface{}{
scheduledPost.Id,
scheduledPost.CreateAt,
scheduledPost.UpdateAt,
scheduledPost.UserId,
scheduledPost.ChannelId,
scheduledPost.RootId,
scheduledPost.Message,
model.StringInterfaceToJSON(scheduledPost.GetProps()),
model.ArrayToJSON(scheduledPost.FileIds),
model.StringInterfaceToJSON(scheduledPost.Priority),
scheduledPost.ScheduledAt,
scheduledPost.ProcessedAt,
scheduledPost.ErrorCode,
}
}
func (s *SqlScheduledPostStore) CreateScheduledPost(scheduledPost *model.ScheduledPost) (*model.ScheduledPost, error) {
scheduledPost.PreSave()
builder := s.getQueryBuilder().
Insert("ScheduledPosts").
Columns(s.columns("")...).
Values(s.scheduledPostToSlice(scheduledPost)...)
query, args, err := builder.ToSql()
if err != nil {
mlog.Error("SqlScheduledPostStore.CreateScheduledPost failed to generate SQL from query builder", mlog.Err(err))
return nil, errors.Wrap(err, "SqlScheduledPostStore.CreateScheduledPost failed to generate SQL from query builder")
}
if _, err := s.GetMasterX().Exec(query, args...); err != nil {
mlog.Error("SqlScheduledPostStore.CreateScheduledPost failed to insert scheduled post", mlog.Err(err))
return nil, errors.Wrap(err, "SqlScheduledPostStore.CreateScheduledPost failed to insert scheduled post")
}
return scheduledPost, nil
}
func (s *SqlScheduledPostStore) GetScheduledPostsForUser(userId, teamId string) ([]*model.ScheduledPost, error) {
// return scheduled posts for this user for
// specified team.
//
//An empty teamId fetches scheduled posts belonging to
// DMs and GMs (DMs and GMs do not belong to any team
// We're intentionally including scheduled posts from archived channels,
// or channels the user no longer belongs to as we want to still show those
// scheduled posts with appropriate error to the user.
// This is why we're not joining with ChannelMembers, and directly
// joining with Channels table.
query := s.getQueryBuilder().
Select(s.columns("sp")...).
From("ScheduledPosts AS sp").
InnerJoin("Channels as c on sp.ChannelId = c.Id").
Where(sq.Eq{
"sp.UserId": userId,
"c.TeamId": teamId,
}).
OrderBy("sp.ScheduledAt, sp.CreateAt")
var scheduledPosts []*model.ScheduledPost
if err := s.GetReplicaX().SelectBuilder(&scheduledPosts, query); err != nil {
mlog.Error("SqlScheduledPostStore.GetScheduledPostsForUser: failed to fetch scheduled posts for user", mlog.String("user_id", userId), mlog.String("team_id", teamId), mlog.Err(err))
return nil, errors.Wrapf(err, "SqlScheduledPostStore.GetScheduledPostsForUser: failed to fetch scheduled posts for user, userId: %s, teamID: %s", userId, teamId)
}
return scheduledPosts, nil
}
func (s *SqlScheduledPostStore) GetMaxMessageSize() int {
s.maxMessageSizeOnce.Do(func() {
var err error
s.maxMessageSizeCached, err = s.SqlStore.determineMaxColumnSize("ScheduledPosts", "Message")
if err != nil {
mlog.Error("SqlScheduledPostStore.getMaxMessageSize: error occurred during determining max column size for ScheduledPosts.Message column", mlog.Err(err))
return
}
})
return s.maxMessageSizeCached
}
func (s *SqlScheduledPostStore) GetPendingScheduledPosts(beforeTime, afterTime int64, lastScheduledPostId string, perPage uint64) ([]*model.ScheduledPost, error) {
query := s.getQueryBuilder().
Select(s.columns("")...).
From("ScheduledPosts").
Where(sq.Eq{"ErrorCode": ""}).
OrderBy("ScheduledAt DESC", "Id").
Limit(perPage)
if lastScheduledPostId == "" {
query = query.Where(sq.And{
sq.LtOrEq{"ScheduledAt": beforeTime},
sq.GtOrEq{"ScheduledAt": afterTime},
})
}
if lastScheduledPostId != "" {
query = query.
Where(sq.Or{
sq.And{
sq.LtOrEq{"ScheduledAt": beforeTime},
sq.GtOrEq{"ScheduledAt": afterTime},
},
sq.And{
sq.Eq{"ScheduledAt": beforeTime},
sq.Gt{"Id": lastScheduledPostId},
},
})
}
var scheduledPosts []*model.ScheduledPost
if err := s.GetReplicaX().SelectBuilder(&scheduledPosts, query); err != nil {
mlog.Error(
"SqlScheduledPostStore.GetPendingScheduledPosts: failed to fetch pending scheduled posts for processing",
mlog.Int("before_time", beforeTime),
mlog.String("last_scheduled_post_id", lastScheduledPostId),
mlog.Uint("items_per_page", perPage), mlog.Err(err),
)
return nil, errors.Wrapf(
err,
"SqlScheduledPostStore.GetPendingScheduledPosts: failed to fetch pending scheduled posts for processing, before_time: %d, last_scheduled_post_id: %s, items_per_page: %d",
beforeTime, lastScheduledPostId, perPage,
)
}
return scheduledPosts, nil
}
func (s *SqlScheduledPostStore) PermanentlyDeleteScheduledPosts(scheduledPostIDs []string) error {
if len(scheduledPostIDs) == 0 {
return nil
}
query := s.getQueryBuilder().
Delete("ScheduledPosts").
Where(sq.Eq{"Id": scheduledPostIDs})
sql, params, err := query.ToSql()
if err != nil {
errToReturn := errors.Wrapf(err, "PermanentlyDeleteScheduledPosts: failed to generate SQL query for permanently deleting batch of scheduled posts")
s.Logger().Error(errToReturn.Error())
return errToReturn
}
if _, err := s.GetMasterX().Exec(sql, params...); err != nil {
errToReturn := errors.Wrapf(err, "PermanentlyDeleteScheduledPosts: failed to delete batch of scheduled posts from database")
s.Logger().Error(errToReturn.Error())
return errToReturn
}
return nil
}
func (s *SqlScheduledPostStore) UpdatedScheduledPost(scheduledPost *model.ScheduledPost) error {
scheduledPost.PreUpdate()
builder := s.getQueryBuilder().
Update("ScheduledPosts").
SetMap(s.toUpdateMap(scheduledPost)).
Where(sq.Eq{"Id": scheduledPost.Id})
query, args, err := builder.ToSql()
if err != nil {
mlog.Error("SqlScheduledPostStore.UpdatedScheduledPost failed to generate SQL from updating scheduled posts", mlog.String("scheduled_post_id", scheduledPost.Id), mlog.Err(err))
return errors.Wrap(err, "SqlScheduledPostStore.UpdatedScheduledPost failed to generate SQL from bulk updating scheduled posts")
}
_, err = s.GetMasterX().Exec(query, args...)
if err != nil {
mlog.Error("SqlScheduledPostStore.UpdatedScheduledPost failed to update scheduled post", mlog.String("scheduled_post_id", scheduledPost.Id), mlog.Err(err))
return errors.Wrap(err, "SqlScheduledPostStore.UpdatedScheduledPost failed to update scheduled post")
}
return nil
}
func (s *SqlScheduledPostStore) toUpdateMap(scheduledPost *model.ScheduledPost) map[string]interface{} {
now := model.GetMillis()
return map[string]interface{}{
"UpdateAt": now,
"Message": scheduledPost.Message,
"Props": model.StringInterfaceToJSON(scheduledPost.GetProps()),
"FileIds": model.ArrayToJSON(scheduledPost.FileIds),
"Priority": model.StringInterfaceToJSON(scheduledPost.Priority),
"ScheduledAt": scheduledPost.ScheduledAt,
"ProcessedAt": now,
"ErrorCode": scheduledPost.ErrorCode,
}
}
func (s *SqlScheduledPostStore) Get(scheduledPostId string) (*model.ScheduledPost, error) {
query := s.getQueryBuilder().
Select(s.columns("")...).
From("ScheduledPosts").
Where(sq.Eq{
"Id": scheduledPostId,
})
scheduledPost := &model.ScheduledPost{}
if err := s.GetReplicaX().GetBuilder(scheduledPost, query); err != nil {
mlog.Error("SqlScheduledPostStore.Get: failed to get single scheduled post by ID from database", mlog.String("scheduled_post_id", scheduledPostId), mlog.Err(err))
return nil, errors.Wrapf(err, "SqlScheduledPostStore.Get: failed to get single scheduled post by ID from database, scheduledPostId: %s", scheduledPostId)
}
return scheduledPost, nil
}
func (s *SqlScheduledPostStore) UpdateOldScheduledPosts(beforeTime int64) error {
builder := s.getQueryBuilder().
Update("ScheduledPosts").
Set("ErrorCode", model.ScheduledPostErrorUnableToSend).
Set("ProcessedAt", model.GetMillis()).
Where(sq.And{
sq.Eq{"ErrorCode": ""},
sq.Lt{"ScheduledAt": beforeTime},
})
query, args, err := builder.ToSql()
if err != nil {
mlog.Error("SqlScheduledPostStore.UpdateOldScheduledPosts failed to generate SQL from updating old scheduled posts", mlog.Err(err))
return errors.Wrap(err, "SqlScheduledPostStore.UpdateOldScheduledPosts failed to generate SQL from updating old scheduled posts")
}
_, err = s.GetMasterX().Exec(query, args...)
if err != nil {
mlog.Error("SqlScheduledPostStore.UpdateOldScheduledPosts failed to update old scheduled posts", mlog.Err(err))
return errors.Wrap(err, "SqlScheduledPostStore.UpdateOldScheduledPosts failed to update old scheduled posts")
}
return nil
}

Просмотреть файл

@@ -0,0 +1,14 @@
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
// See LICENSE.txt for license information.
package sqlstore
import (
"testing"
"github.com/mattermost/mattermost/server/v8/channels/store/storetest"
)
func TestScheduledPostStore(t *testing.T) {
StoreTestWithSqlStore(t, storetest.TestScheduledPostStore)
}

Просмотреть файл

@@ -111,6 +111,7 @@ type SqlStoreStores struct {
postPersistentNotification store.PostPersistentNotificationStore
desktopTokens store.DesktopTokensStore
channelBookmarks store.ChannelBookmarkStore
scheduledPost store.ScheduledPostStore
}
type SqlStore struct {
@@ -236,6 +237,7 @@ func New(settings model.SqlSettings, logger mlog.LoggerIFace, metrics einterface
store.stores.postPersistentNotification = newSqlPostPersistentNotificationStore(store)
store.stores.desktopTokens = newSqlDesktopTokensStore(store, metrics)
store.stores.channelBookmarks = newSqlChannelBookmarkStore(store)
store.stores.scheduledPost = newScheduledPostStore(store)
store.stores.preference.(*SqlPreferenceStore).deleteUnusedFeatures()
@@ -1294,3 +1296,51 @@ func (ss *SqlStore) GetAppliedMigrations() ([]model.AppliedMigration, error) {
return migrations, nil
}
func (ss *SqlStore) determineMaxColumnSize(tableName, columnName string) (int, error) {
var columnSizeBytes int32
ss.getQueryPlaceholder()
if ss.DriverName() == model.DatabaseDriverPostgres {
if err := ss.GetReplicaX().Get(&columnSizeBytes, `
SELECT
COALESCE(character_maximum_length, 0)
FROM
information_schema.columns
WHERE
lower(table_name) = lower($1)
AND lower(column_name) = lower($2)
`, tableName, columnName); err != nil {
mlog.Warn("Unable to determine the maximum supported column size for Postgres", mlog.Err(err))
return 0, err
}
} else if ss.DriverName() == model.DatabaseDriverMysql {
if err := ss.GetReplicaX().Get(&columnSizeBytes, `
SELECT
COALESCE(CHARACTER_MAXIMUM_LENGTH, 0)
FROM
INFORMATION_SCHEMA.COLUMNS
WHERE
table_schema = DATABASE()
AND lower(table_name) = lower(?)
AND lower(column_name) = lower(?)
LIMIT 0, 1
`, tableName, columnName); err != nil {
mlog.Warn("Unable to determine the maximum supported column size for MySQL", mlog.Err(err))
return 0, err
}
} else {
mlog.Warn("No implementation found to determine the maximum supported column size")
}
// Assume a worst-case representation of four bytes per rune.
maxColumnSize := int(columnSizeBytes) / 4
mlog.Info("Column has size restrictions", mlog.String("table_name", tableName), mlog.String("column_name", columnName), mlog.Int("max_characters", maxColumnSize), mlog.Int("max_bytes", columnSizeBytes))
return maxColumnSize, nil
}
func (ss *SqlStore) ScheduledPost() store.ScheduledPostStore {
return ss.stores.scheduledPost
}