Ldap groups phase1 (#9752)

* Initial models, API, app, and persistence of groups and group syncing. * Consistent letter casing in ldif. * Moves group-specific migrations into func. * Adds API endpoint to retrieve LDAP groups (and associated MM groups) one tree level at a time. * Adds mattermost group id to SCIMGroup (if available). * Splits user and group creation so that memberOf works. Returns users from ldap interface. * Updates method name. * Returns users IDs instead of User. * Removes non-essential group data. * MM-11807: Add GroupFilter to LDAP config. (#9513) * MM-11807: Add GroupFilter to LDAP config. * Add diagnostic. * Adds new config option for using 'memberOf' overlay. * Adds API endpoint to link a group. * Removes debug statements. * Adds unlink group API endpoint. * Fix to LDAP API. Adds API method to client4 and app. * Adds some missing app methods. Renames API unexported func. * Fixes link/unlink API path to accept valid DNs. * Allow any character for DN portion of path. * Switches from DN to objectGUID or entryUUID as the remote identifier linking LDAP groups to MM groups. * Formatting. * Formatting. * Setting group name field to an ID for phase 1. * Adds an LDAP config field to Setting up configuration for local LDAP. * Changes to LDAP and GroupStore interfaces. * Draft of nesting groups in API response. * Removes unnecessary tree models. * Updates group membershipt create store method to also restore. * Adds new config to test config. * Accept AD format length. * Switches to SetUniqueTogether method. * Updates revert. * Tweaks to syncing queries . * Updates query for pending team and channel memberships. * Removes old GroupSyncableScanner usage. Some formatting and renaming. * Fixes bug setting syncable type in selecting paged. * Adds tests for syncables populator. * Only add users to teams and channels that are not deleted. * Renames method. * Updates test LDAP setup. * Removes memberof config stuff. * Renames. * Updates test data. * Fix for gofmt. * Adds missing license. * Adds missing teardowns. * Test fix. * Adds a cycle to the groups test data. * Changes API to return flat list. * Removes some unused interface and app methods. * Returns empty braces if results are empty. * Adds more LDAP test data. * Fix for test data error. * Adds error. * Moves test groups. * Adds OU for load test data. * Moves load test ou creation to load data. * Adds a new bool flag to SCIMGroups. * Removes SCIMGroup completely. * Removes FULL JOIN because it is not supported in MySQL. * Adds tests for sync queries; renames constant. * Bad merge fix. * Vet fix. * Returning OK on delete ldap group link * Removes foreign key constraints. * Adding total to the ldap getAllGroups api endpoint * Adds get group members page. * Removes pagination from groups syncables list API. * Adding syncable check now that foreign key constraint is removes. * Joins teams and channels to group syncables. * Adds group member count. * Adding GetAllChannels and SearchAllChannels for system admins only * Fix. * Test fix from pagination removal. * Orders groupmembers by createat. * Fixing search of all channels * Test fix after removing pagination. * JSON syntax error fix. * Changing tests (for now) pending investigation. * Adding GetAllChannels and SearchAllChannels tests for the store * Adding GetAllChannels and SearchAllChannels API tests * Omit empty JSON values of group syncables. * Fixing GetAllChannels and SearchAllChannels tests * Fixing GetAllChannels and SearchAllChannels store tests * Fixing GetAllChannels api tests * Adds 'LDAP groups' feature flag. (#9861) * Migrate new client functions to idiomatic error handling * Test fixes. * Simplification of groups api (#9860) * Simplification of groups api * Fixing RequireSyncableType * Test fix. * Update api4/group.go Co-Authored-By: mkraft <martinkraft@gmail.com> * Update api4/group.go Co-Authored-By: mkraft <martinkraft@gmail.com> * Update api4/group.go Co-Authored-By: mkraft <martinkraft@gmail.com> * Update api4/group.go Co-Authored-By: mkraft <martinkraft@gmail.com> * Update api4/group.go Co-Authored-By: mkraft <martinkraft@gmail.com> * Update api4/group.go Co-Authored-By: mkraft <martinkraft@gmail.com> * Update api4/group.go Co-Authored-By: mkraft <martinkraft@gmail.com> * Update api4/group.go Co-Authored-By: mkraft <martinkraft@gmail.com> * Update api4/group.go Co-Authored-By: mkraft <martinkraft@gmail.com> * Update api4/group.go Co-Authored-By: mkraft <martinkraft@gmail.com> * Fix copy/paste error. * Fix copy/paste error. * Adds missing return, changes to correct HTTP status code. * Adds missing return, changes status codes. * Check for license. * Renames variable for new signature. * Adds client method to get a group. * Adds client method and tests for PatchGroup. * Adds more API tests. * Adds groups API tests. * Adds client method and tests for getting group syncables. * Adds tests for patching group teams and channels. * Update to translations. * Removes test. * Fix incorrect conditional. * Removes unnecessary nil check. * Removes unnecessary return. * Updates comment, removes unused variable. * Uses consistent JSON unmarshal pattern. * Uses consistent JSON unmarshal pattern. * Moves const block. * Switches 'already linked' from error to success response. * Removes commented-out code. * Switched to status ok. * Add parens for readability. * Fix copy/paste error. * Unexport some structs. * Removes repeated validity check. * Return without attempting commit if there's a rollback. * Fix incorrect HTTP status code. * Update store/sqlstore/group_supplier.go Co-Authored-By: mkraft <martinkraft@gmail.com> * Adds utility methods for going from groupsyncable to groupteam and groupchannel. * Fixing george suggestions (#9911) * Test fix. * Adds QA data to VC with visualization. * Fixes typo in graph image. * Update display name when re-linking in case it has changed in LDAP. * Adds ability to configure group display name and unique identifier. (#9923) * Adds ability to configure group display name and unique identifier. * Adds some configs to confi-ldap make command. * Fix for move of session. * Exposes method for use by SAML package. * Switches GroupSyncableType from int to string. * Update Jenkins build files. * Removes unused variable assignment. * Removes old unnecessary early return. * Removes unnecessary variable. * Moves param parsing before license and permissions checks. * Removes old code. * Compares agains underlying error rather than error id. * Switches tests to assertions. * Adds more assertions. * Adds missing return. * Adds space after comma for added legibility. * Moves a view model to the api package. * Unexports method. * Uses id validator function. * Fix docker-compose flag. * Typo fix. * Moves index creation to supplier. * Removes bad merge. * Renames parameter. * Re-adds space. * Removes unnecessary transaction. * Escapes the Groups table name with backticks because it is a reserved keyword. * Fix roles cache bug * Removing unnecesiary deserializing function * Switches table name rather than custom SQL everywhere for Postgres without backticks. * Removes redundant check for sql.ErrNoRows. * Removes redundant check for sql.ErrNoRows. * Removes data integrity check and redundant nil conditional. * Removes redundant check for sql.ErrNoRows. * Removes unnecessary query. * Removes ID length validation from persistence tier. * Makes some supplier methods idempotent. * Removes some empty switch defaults. * Renames Group Type field to Source. * Fix for mistaken field name change. * Uses IsValidId function. * Removes comment. * Changes json key name. * Removes test because no longer validating user. * Moves model state validation to app layer. * Don't create Groups.CanLeave column until phase 2. * Removes state validation until properties are used in phase 2. * Removes duplicated check. * Removes state validation until properties are used in phase 2. * Removes some tests until phase 2. * Comment-out a bunch of test related to CanLeave. * Extra unmarshal validation check. Removes more code for CanLeave. * Removes tests for CanLeave. * Explict error msg. * Rewrite queries. * Changes index name. Adds index. * Removes assertion. * Adds experimental feature flag.
2019-01-10 15:17:31 -05:00
--- a/app/channel.go
+++ b/app/channel.go
@@ -1116,6 +1116,14 @@ func (a *App) GetChannelsForUser(teamId string, userId string, includeDeleted bo
 	return result.Data.(*model.ChannelList), nil
 }

+func (a *App) GetAllChannels(page, perPage int, includeDeleted bool) (*model.ChannelListWithTeamData, *model.AppError) {
+	result := <-a.Srv.Store.Channel().GetAllChannels(page*perPage, perPage, includeDeleted)
+	if result.Err != nil {
+		return nil, result.Err
+	}
+	return result.Data.(*model.ChannelListWithTeamData), nil
+}
+
 func (a *App) GetDeletedChannels(teamId string, offset int, limit int) (*model.ChannelList, *model.AppError) {
 	result := <-a.Srv.Store.Channel().GetDeleted(teamId, offset, limit)
 	if result.Err != nil {
@@ -1577,6 +1585,14 @@ func (a *App) AutocompleteChannelsForSearch(teamId string, userId string, term s
 	return result.Data.(*model.ChannelList), nil
 }

+func (a *App) SearchAllChannels(term string, includeDeleted bool) (*model.ChannelListWithTeamData, *model.AppError) {
+	result := <-a.Srv.Store.Channel().SearchAllChannels(term, *a.Config().TeamSettings.ExperimentalViewArchivedChannels && includeDeleted)
+	if result.Err != nil {
+		return nil, result.Err
+	}
+	return result.Data.(*model.ChannelListWithTeamData), nil
+}
+
 func (a *App) SearchChannels(teamId string, term string) (*model.ChannelList, *model.AppError) {
 	includeDeleted := *a.Config().TeamSettings.ExperimentalViewArchivedChannels

--- a/app/diagnostics.go
+++ b/app/diagnostics.go
@@ -264,6 +264,7 @@ func (a *App) trackConfig() {
 		"experimental_enable_hardened_mode":                       *cfg.ServiceSettings.ExperimentalEnableHardenedMode,
 		"enable_email_invitations":                                *cfg.ServiceSettings.EnableEmailInvitations,
 		"experimental_channel_organization":                       *cfg.ServiceSettings.ExperimentalChannelOrganization,
+		"experimental_ldap_group_sync":                            *cfg.ServiceSettings.ExperimentalLdapGroupSync,
 	})

 	a.SendDiagnostic(TRACK_CONFIG_TEAM, map[string]interface{}{
@@ -418,25 +419,28 @@ func (a *App) trackConfig() {
 	})

 	a.SendDiagnostic(TRACK_CONFIG_LDAP, map[string]interface{}{
-		"enable":                              *cfg.LdapSettings.Enable,
-		"enable_sync":                         *cfg.LdapSettings.EnableSync,
-		"connection_security":                 *cfg.LdapSettings.ConnectionSecurity,
-		"skip_certificate_verification":       *cfg.LdapSettings.SkipCertificateVerification,
-		"sync_interval_minutes":               *cfg.LdapSettings.SyncIntervalMinutes,
-		"query_timeout":                       *cfg.LdapSettings.QueryTimeout,
-		"max_page_size":                       *cfg.LdapSettings.MaxPageSize,
-		"isdefault_first_name_attribute":      isDefault(*cfg.LdapSettings.FirstNameAttribute, model.LDAP_SETTINGS_DEFAULT_FIRST_NAME_ATTRIBUTE),
-		"isdefault_last_name_attribute":       isDefault(*cfg.LdapSettings.LastNameAttribute, model.LDAP_SETTINGS_DEFAULT_LAST_NAME_ATTRIBUTE),
-		"isdefault_email_attribute":           isDefault(*cfg.LdapSettings.EmailAttribute, model.LDAP_SETTINGS_DEFAULT_EMAIL_ATTRIBUTE),
-		"isdefault_username_attribute":        isDefault(*cfg.LdapSettings.UsernameAttribute, model.LDAP_SETTINGS_DEFAULT_USERNAME_ATTRIBUTE),
-		"isdefault_nickname_attribute":        isDefault(*cfg.LdapSettings.NicknameAttribute, model.LDAP_SETTINGS_DEFAULT_NICKNAME_ATTRIBUTE),
-		"isdefault_id_attribute":              isDefault(*cfg.LdapSettings.IdAttribute, model.LDAP_SETTINGS_DEFAULT_ID_ATTRIBUTE),
-		"isdefault_position_attribute":        isDefault(*cfg.LdapSettings.PositionAttribute, model.LDAP_SETTINGS_DEFAULT_POSITION_ATTRIBUTE),
-		"isdefault_login_id_attribute":        isDefault(*cfg.LdapSettings.LoginIdAttribute, ""),
-		"isdefault_login_field_name":          isDefault(*cfg.LdapSettings.LoginFieldName, model.LDAP_SETTINGS_DEFAULT_LOGIN_FIELD_NAME),
-		"isdefault_login_button_color":        isDefault(*cfg.LdapSettings.LoginButtonColor, ""),
-		"isdefault_login_button_border_color": isDefault(*cfg.LdapSettings.LoginButtonBorderColor, ""),
-		"isdefault_login_button_text_color":   isDefault(*cfg.LdapSettings.LoginButtonTextColor, ""),
+		"enable":                                 *cfg.LdapSettings.Enable,
+		"enable_sync":                            *cfg.LdapSettings.EnableSync,
+		"connection_security":                    *cfg.LdapSettings.ConnectionSecurity,
+		"skip_certificate_verification":          *cfg.LdapSettings.SkipCertificateVerification,
+		"sync_interval_minutes":                  *cfg.LdapSettings.SyncIntervalMinutes,
+		"query_timeout":                          *cfg.LdapSettings.QueryTimeout,
+		"max_page_size":                          *cfg.LdapSettings.MaxPageSize,
+		"isdefault_first_name_attribute":         isDefault(*cfg.LdapSettings.FirstNameAttribute, model.LDAP_SETTINGS_DEFAULT_FIRST_NAME_ATTRIBUTE),
+		"isdefault_last_name_attribute":          isDefault(*cfg.LdapSettings.LastNameAttribute, model.LDAP_SETTINGS_DEFAULT_LAST_NAME_ATTRIBUTE),
+		"isdefault_email_attribute":              isDefault(*cfg.LdapSettings.EmailAttribute, model.LDAP_SETTINGS_DEFAULT_EMAIL_ATTRIBUTE),
+		"isdefault_username_attribute":           isDefault(*cfg.LdapSettings.UsernameAttribute, model.LDAP_SETTINGS_DEFAULT_USERNAME_ATTRIBUTE),
+		"isdefault_nickname_attribute":           isDefault(*cfg.LdapSettings.NicknameAttribute, model.LDAP_SETTINGS_DEFAULT_NICKNAME_ATTRIBUTE),
+		"isdefault_id_attribute":                 isDefault(*cfg.LdapSettings.IdAttribute, model.LDAP_SETTINGS_DEFAULT_ID_ATTRIBUTE),
+		"isdefault_position_attribute":           isDefault(*cfg.LdapSettings.PositionAttribute, model.LDAP_SETTINGS_DEFAULT_POSITION_ATTRIBUTE),
+		"isdefault_login_id_attribute":           isDefault(*cfg.LdapSettings.LoginIdAttribute, ""),
+		"isdefault_login_field_name":             isDefault(*cfg.LdapSettings.LoginFieldName, model.LDAP_SETTINGS_DEFAULT_LOGIN_FIELD_NAME),
+		"isdefault_login_button_color":           isDefault(*cfg.LdapSettings.LoginButtonColor, ""),
+		"isdefault_login_button_border_color":    isDefault(*cfg.LdapSettings.LoginButtonBorderColor, ""),
+		"isdefault_login_button_text_color":      isDefault(*cfg.LdapSettings.LoginButtonTextColor, ""),
+		"isempty_group_filter":                   isDefault(*cfg.LdapSettings.GroupFilter, ""),
+		"isdefault_group_display_name_attribute": isDefault(*cfg.LdapSettings.GroupDisplayNameAttribute, model.LDAP_SETTINGS_DEFAULT_GROUP_DISPLAY_NAME_ATTRIBUTE),
+		"isdefault_group_id_attribute":           isDefault(*cfg.LdapSettings.GroupIdAttribute, model.LDAP_SETTINGS_DEFAULT_GROUP_ID_ATTRIBUTE),
 	})

 	a.SendDiagnostic(TRACK_CONFIG_COMPLIANCE, map[string]interface{}{
--- a/app/group.go
+++ b/app/group.go
@@ -0,0 +1,150 @@
+// Copyright (c) 2018-present Mattermost, Inc. All Rights Reserved.
+// See License.txt for license information.
+
+package app
+
+import (
+	"github.com/mattermost/mattermost-server/model"
+)
+
+func (a *App) GetGroup(id string) (*model.Group, *model.AppError) {
+	result := <-a.Srv.Store.Group().Get(id)
+	if result.Err != nil {
+		return nil, result.Err
+	}
+	return result.Data.(*model.Group), nil
+}
+
+func (a *App) GetGroupByRemoteID(remoteID string, groupSource model.GroupSource) (*model.Group, *model.AppError) {
+	result := <-a.Srv.Store.Group().GetByRemoteID(remoteID, groupSource)
+	if result.Err != nil {
+		return nil, result.Err
+	}
+	return result.Data.(*model.Group), nil
+}
+
+func (a *App) GetGroupsBySource(groupSource model.GroupSource) ([]*model.Group, *model.AppError) {
+	result := <-a.Srv.Store.Group().GetAllBySource(groupSource)
+	if result.Err != nil {
+		return nil, result.Err
+	}
+	return result.Data.([]*model.Group), nil
+}
+
+func (a *App) CreateGroup(group *model.Group) (*model.Group, *model.AppError) {
+	result := <-a.Srv.Store.Group().Create(group)
+	if result.Err != nil {
+		return nil, result.Err
+	}
+	return result.Data.(*model.Group), nil
+}
+
+func (a *App) UpdateGroup(group *model.Group) (*model.Group, *model.AppError) {
+	result := <-a.Srv.Store.Group().Update(group)
+	if result.Err != nil {
+		return nil, result.Err
+	}
+	return result.Data.(*model.Group), nil
+}
+
+func (a *App) DeleteGroup(groupID string) (*model.Group, *model.AppError) {
+	result := <-a.Srv.Store.Group().Delete(groupID)
+	if result.Err != nil {
+		return nil, result.Err
+	}
+	return result.Data.(*model.Group), nil
+}
+
+func (a *App) GetGroupMemberUsers(groupID string) ([]*model.User, *model.AppError) {
+	result := <-a.Srv.Store.Group().GetMemberUsers(groupID)
+	if result.Err != nil {
+		return nil, result.Err
+	}
+	return result.Data.([]*model.User), nil
+}
+
+func (a *App) GetGroupMemberUsersPage(groupID string, page int, perPage int) ([]*model.User, int, *model.AppError) {
+	result := <-a.Srv.Store.Group().GetMemberUsersPage(groupID, page, perPage)
+	if result.Err != nil {
+		return nil, 0, result.Err
+	}
+	members := result.Data.([]*model.User)
+	result = <-a.Srv.Store.Group().GetMemberCount(groupID)
+	if result.Err != nil {
+		return nil, 0, result.Err
+	}
+	count := int(result.Data.(int64))
+	return members, count, nil
+}
+
+func (a *App) CreateOrRestoreGroupMember(groupID string, userID string) (*model.GroupMember, *model.AppError) {
+	result := <-a.Srv.Store.Group().CreateOrRestoreMember(groupID, userID)
+	if result.Err != nil {
+		return nil, result.Err
+	}
+	return result.Data.(*model.GroupMember), nil
+}
+
+func (a *App) DeleteGroupMember(groupID string, userID string) (*model.GroupMember, *model.AppError) {
+	result := <-a.Srv.Store.Group().DeleteMember(groupID, userID)
+	if result.Err != nil {
+		return nil, result.Err
+	}
+	return result.Data.(*model.GroupMember), nil
+}
+
+func (a *App) CreateGroupSyncable(groupSyncable *model.GroupSyncable) (*model.GroupSyncable, *model.AppError) {
+	result := <-a.Srv.Store.Group().CreateGroupSyncable(groupSyncable)
+	if result.Err != nil {
+		return nil, result.Err
+	}
+	return result.Data.(*model.GroupSyncable), nil
+}
+
+func (a *App) GetGroupSyncable(groupID string, syncableID string, syncableType model.GroupSyncableType) (*model.GroupSyncable, *model.AppError) {
+	result := <-a.Srv.Store.Group().GetGroupSyncable(groupID, syncableID, syncableType)
+	if result.Err != nil {
+		return nil, result.Err
+	}
+	return result.Data.(*model.GroupSyncable), nil
+}
+
+func (a *App) GetGroupSyncables(groupID string, syncableType model.GroupSyncableType) ([]*model.GroupSyncable, *model.AppError) {
+	result := <-a.Srv.Store.Group().GetAllGroupSyncablesByGroupId(groupID, syncableType)
+	if result.Err != nil {
+		return nil, result.Err
+	}
+	return result.Data.([]*model.GroupSyncable), nil
+}
+
+func (a *App) UpdateGroupSyncable(groupSyncable *model.GroupSyncable) (*model.GroupSyncable, *model.AppError) {
+	result := <-a.Srv.Store.Group().UpdateGroupSyncable(groupSyncable)
+	if result.Err != nil {
+		return nil, result.Err
+	}
+	return result.Data.(*model.GroupSyncable), nil
+}
+
+func (a *App) DeleteGroupSyncable(groupID string, syncableID string, syncableType model.GroupSyncableType) (*model.GroupSyncable, *model.AppError) {
+	result := <-a.Srv.Store.Group().DeleteGroupSyncable(groupID, syncableID, syncableType)
+	if result.Err != nil {
+		return nil, result.Err
+	}
+	return result.Data.(*model.GroupSyncable), nil
+}
+
+func (a *App) PendingAutoAddTeamMembers(minGroupMembersCreateAt int64) ([]*model.UserTeamIDPair, *model.AppError) {
+	result := <-a.Srv.Store.Group().PendingAutoAddTeamMembers(minGroupMembersCreateAt)
+	if result.Err != nil {
+		return nil, result.Err
+	}
+	return result.Data.([]*model.UserTeamIDPair), nil
+}
+
+func (a *App) PendingAutoAddChannelMembers(minGroupMembersCreateAt int64) ([]*model.UserChannelIDPair, *model.AppError) {
+	result := <-a.Srv.Store.Group().PendingAutoAddChannelMembers(minGroupMembersCreateAt)
+	if result.Err != nil {
+		return nil, result.Err
+	}
+	return result.Data.([]*model.UserChannelIDPair), nil
+}
--- a/app/group_test.go
+++ b/app/group_test.go
@@ -0,0 +1,223 @@
+// Copyright (c) 2016-present Mattermost, Inc. All Rights Reserved.
+// See License.txt for license information.
+
+package app
+
+import (
+	"testing"
+
+	"github.com/mattermost/mattermost-server/model"
+	"github.com/stretchr/testify/require"
+)
+
+func TestGetGroup(t *testing.T) {
+	th := Setup().InitBasic()
+	defer th.TearDown()
+	group := th.CreateGroup()
+
+	group, err := th.App.GetGroup(group.Id)
+	require.Nil(t, err)
+	require.NotNil(t, group)
+
+	group, err = th.App.GetGroup(model.NewId())
+	require.NotNil(t, err)
+	require.Nil(t, group)
+}
+
+func TestGetGroupByRemoteID(t *testing.T) {
+	th := Setup().InitBasic()
+	defer th.TearDown()
+	group := th.CreateGroup()
+
+	g, err := th.App.GetGroupByRemoteID(group.RemoteId, model.GroupSourceLdap)
+	require.Nil(t, err)
+	require.NotNil(t, g)
+
+	g, err = th.App.GetGroupByRemoteID(model.NewId(), model.GroupSourceLdap)
+	require.NotNil(t, err)
+	require.Nil(t, g)
+}
+
+func TestGetGroupsByType(t *testing.T) {
+	th := Setup().InitBasic()
+	defer th.TearDown()
+	th.CreateGroup()
+	th.CreateGroup()
+	th.CreateGroup()
+
+	groups, err := th.App.GetGroupsBySource(model.GroupSourceLdap)
+	require.Nil(t, err)
+	require.NotEmpty(t, groups)
+
+	groups, err = th.App.GetGroupsBySource(model.GroupSource("blah"))
+	require.Nil(t, err)
+	require.Empty(t, groups)
+}
+
+func TestCreateGroup(t *testing.T) {
+	th := Setup().InitBasic()
+	defer th.TearDown()
+
+	id := model.NewId()
+	group := &model.Group{
+		DisplayName: "dn_" + id,
+		Name:        "name" + id,
+		Source:      model.GroupSourceLdap,
+		RemoteId:    model.NewId(),
+	}
+
+	g, err := th.App.CreateGroup(group)
+	require.Nil(t, err)
+	require.NotNil(t, g)
+
+	g, err = th.App.CreateGroup(group)
+	require.NotNil(t, err)
+	require.Nil(t, g)
+}
+
+func TestUpdateGroup(t *testing.T) {
+	th := Setup().InitBasic()
+	defer th.TearDown()
+	group := th.CreateGroup()
+	group.DisplayName = model.NewId()
+
+	g, err := th.App.UpdateGroup(group)
+	require.Nil(t, err)
+	require.NotNil(t, g)
+}
+
+func TestDeleteGroup(t *testing.T) {
+	th := Setup().InitBasic()
+	defer th.TearDown()
+	group := th.CreateGroup()
+
+	g, err := th.App.DeleteGroup(group.Id)
+	require.Nil(t, err)
+	require.NotNil(t, g)
+
+	g, err = th.App.DeleteGroup(group.Id)
+	require.NotNil(t, err)
+	require.Nil(t, g)
+}
+
+func TestCreateOrRestoreGroupMember(t *testing.T) {
+	th := Setup().InitBasic()
+	defer th.TearDown()
+	group := th.CreateGroup()
+
+	g, err := th.App.CreateOrRestoreGroupMember(group.Id, th.BasicUser.Id)
+	require.Nil(t, err)
+	require.NotNil(t, g)
+
+	g, err = th.App.CreateOrRestoreGroupMember(group.Id, th.BasicUser.Id)
+	require.NotNil(t, err)
+	require.Nil(t, g)
+}
+
+func TestDeleteGroupMember(t *testing.T) {
+	th := Setup().InitBasic()
+	defer th.TearDown()
+	group := th.CreateGroup()
+	groupMember, err := th.App.CreateOrRestoreGroupMember(group.Id, th.BasicUser.Id)
+	require.Nil(t, err)
+	require.NotNil(t, groupMember)
+
+	groupMember, err = th.App.DeleteGroupMember(groupMember.GroupId, groupMember.UserId)
+	require.Nil(t, err)
+	require.NotNil(t, groupMember)
+
+	groupMember, err = th.App.DeleteGroupMember(groupMember.GroupId, groupMember.UserId)
+	require.NotNil(t, err)
+	require.Nil(t, groupMember)
+}
+
+func TestCreateGroupSyncable(t *testing.T) {
+	th := Setup().InitBasic()
+	defer th.TearDown()
+	group := th.CreateGroup()
+	groupSyncable := &model.GroupSyncable{
+		GroupId:    group.Id,
+		CanLeave:   true,
+		AutoAdd:    false,
+		SyncableId: th.BasicTeam.Id,
+		Type:       model.GroupSyncableTypeTeam,
+	}
+
+	gs, err := th.App.CreateGroupSyncable(groupSyncable)
+	require.Nil(t, err)
+	require.NotNil(t, gs)
+
+	gs, err = th.App.CreateGroupSyncable(groupSyncable)
+	require.NotNil(t, err)
+	require.Nil(t, gs)
+}
+
+func TestGetGroupSyncable(t *testing.T) {
+	th := Setup().InitBasic()
+	defer th.TearDown()
+	group := th.CreateGroup()
+	groupSyncable := &model.GroupSyncable{
+		GroupId:    group.Id,
+		CanLeave:   true,
+		AutoAdd:    false,
+		SyncableId: th.BasicTeam.Id,
+		Type:       model.GroupSyncableTypeTeam,
+	}
+
+	gs, err := th.App.CreateGroupSyncable(groupSyncable)
+	require.Nil(t, err)
+	require.NotNil(t, gs)
+
+	gs, err = th.App.GetGroupSyncable(group.Id, th.BasicTeam.Id, model.GroupSyncableTypeTeam)
+	require.Nil(t, err)
+	require.NotNil(t, gs)
+}
+
+func TestGetGroupSyncables(t *testing.T) {
+	th := Setup().InitBasic()
+	defer th.TearDown()
+	group := th.CreateGroup()
+
+	// Create a group team
+	groupSyncable := &model.GroupSyncable{
+		GroupId:    group.Id,
+		CanLeave:   true,
+		AutoAdd:    false,
+		SyncableId: th.BasicTeam.Id,
+		Type:       model.GroupSyncableTypeTeam,
+	}
+
+	gs, err := th.App.CreateGroupSyncable(groupSyncable)
+	require.Nil(t, err)
+	require.NotNil(t, gs)
+
+	groupTeams, err := th.App.GetGroupSyncables(group.Id, model.GroupSyncableTypeTeam)
+	require.Nil(t, err)
+
+	require.NotEmpty(t, groupTeams)
+}
+
+func TestDeleteGroupSyncable(t *testing.T) {
+	th := Setup().InitBasic()
+	defer th.TearDown()
+	group := th.CreateGroup()
+	groupChannel := &model.GroupSyncable{
+		GroupId:    group.Id,
+		CanLeave:   true,
+		AutoAdd:    false,
+		SyncableId: th.BasicChannel.Id,
+		Type:       model.GroupSyncableTypeChannel,
+	}
+
+	gs, err := th.App.CreateGroupSyncable(groupChannel)
+	require.Nil(t, err)
+	require.NotNil(t, gs)
+
+	gs, err = th.App.DeleteGroupSyncable(group.Id, th.BasicChannel.Id, model.GroupSyncableTypeChannel)
+	require.Nil(t, err)
+	require.NotNil(t, gs)
+
+	gs, err = th.App.DeleteGroupSyncable(group.Id, th.BasicChannel.Id, model.GroupSyncableTypeChannel)
+	require.NotNil(t, err)
+	require.Nil(t, gs)
+}
--- a/app/helper_test.go
+++ b/app/helper_test.go
@@ -343,6 +343,28 @@ func (me *TestHelper) CreateScheme() (*model.Scheme, []*model.Role) {
 	return scheme, roles
 }

+func (me *TestHelper) CreateGroup() *model.Group {
+	id := model.NewId()
+	group := &model.Group{
+		DisplayName: "dn_" + id,
+		Name:        "name" + id,
+		Source:      model.GroupSourceLdap,
+		Description: "description_" + id,
+		RemoteId:    model.NewId(),
+	}
+
+	utils.DisableDebugLogForTest()
+	var err *model.AppError
+	if group, err = me.App.CreateGroup(group); err != nil {
+		mlog.Error(err.Error())
+
+		time.Sleep(time.Second)
+		panic(err)
+	}
+	utils.EnableDebugLogForTest()
+	return group
+}
+
 func (me *TestHelper) CreateEmoji() *model.Emoji {
 	utils.DisableDebugLogForTest()

--- a/app/ldap.go
+++ b/app/ldap.go
@@ -40,6 +40,46 @@ func (a *App) TestLdap() *model.AppError {
 	return nil
 }

+// GetLdapGroup retrieves a single LDAP group by the given LDAP group id.
+func (a *App) GetLdapGroup(ldapGroupID string) (*model.Group, *model.AppError) {
+	var group *model.Group
+
+	if a.Ldap != nil {
+		var err *model.AppError
+		group, err = a.Ldap.GetGroup(ldapGroupID)
+		if err != nil {
+			return nil, err
+		}
+	} else {
+		ae := model.NewAppError("GetLdapGroup", "ent.ldap.app_error", nil, "", http.StatusNotImplemented)
+		mlog.Error(fmt.Sprintf("%v", ae.Error()))
+		return nil, ae
+	}
+
+	return group, nil
+}
+
+// GetAllLdapGroupsPage retrieves all LDAP groups under the configured base DN using the default or configured group
+// filter.
+func (a *App) GetAllLdapGroupsPage(page int, perPage int) ([]*model.Group, int, *model.AppError) {
+	var groups []*model.Group
+	var total int
+
+	if a.Ldap != nil {
+		var err *model.AppError
+		groups, total, err = a.Ldap.GetAllGroupsPage(page, perPage)
+		if err != nil {
+			return nil, 0, err
+		}
+	} else {
+		ae := model.NewAppError("GetAllLdapGroupsPage", "ent.ldap.app_error", nil, "", http.StatusNotImplemented)
+		mlog.Error(fmt.Sprintf("%v", ae.Error()))
+		return nil, 0, ae
+	}
+
+	return groups, total, nil
+}
+
 func (a *App) SwitchEmailToLdap(email, password, code, ldapLoginId, ldapPassword string) (string, *model.AppError) {
 	if a.License() != nil && !*a.Config().ServiceSettings.ExperimentalEnableAuthenticationTransfer {
 		return "", model.NewAppError("emailToLdap", "api.user.email_to_ldap.not_available.app_error", nil, "", http.StatusForbidden)
--- a/app/syncables.go
+++ b/app/syncables.go
@@ -0,0 +1,63 @@
+// Copyright (c) 2016-present Mattermost, Inc. All Rights Reserved.
+// See License.txt for license information.
+
+package app
+
+import (
+	"github.com/mattermost/mattermost-server/mlog"
+)
+
+// PopulateSyncablesSince adds users to teams and channels based on their group memberships and how those groups are
+// configured to sync with teams and channels for group members on or after the given timestamp.
+func (a *App) PopulateSyncablesSince(groupMembersCreatedAfter int64) error {
+	userTeamIDs, appErr := a.PendingAutoAddTeamMembers(groupMembersCreatedAfter)
+	if appErr != nil {
+		return appErr
+	}
+
+	for _, userTeam := range userTeamIDs {
+		_, err := a.AddTeamMember(userTeam.TeamID, userTeam.UserID)
+		if err != nil {
+			return err
+		}
+
+		a.Log.Info("added teammember",
+			mlog.String("user_id", userTeam.UserID),
+			mlog.String("team_id", userTeam.TeamID),
+		)
+	}
+
+	userChannelIDs, appErr := a.PendingAutoAddChannelMembers(groupMembersCreatedAfter)
+	if appErr != nil {
+		return appErr
+	}
+
+	for _, userChannel := range userChannelIDs {
+		channel, err := a.GetChannel(userChannel.ChannelID)
+		if err != nil {
+			return err
+		}
+
+		// First add user to team
+		_, err = a.AddTeamMember(channel.TeamId, userChannel.UserID)
+		if err != nil {
+			return err
+		}
+		a.Log.Info("added teammember",
+			mlog.String("user_id", userChannel.UserID),
+			mlog.String("team_id", channel.TeamId),
+		)
+
+		_, err = a.AddChannelMember(userChannel.UserID, channel, "", "", false)
+		if err != nil {
+			return err
+		}
+
+		a.Log.Info("added channelmember",
+			mlog.String("user_id", userChannel.UserID),
+			mlog.String("channel_id", userChannel.ChannelID),
+		)
+	}
+
+	return nil
+}
--- a/app/syncables_test.go
+++ b/app/syncables_test.go
@@ -0,0 +1,346 @@
+package app
+
+import (
+	"testing"
+
+	"github.com/mattermost/mattermost-server/model"
+)
+
+func TestPopulateSyncablesSince(t *testing.T) {
+	th := Setup().InitBasic()
+	defer th.TearDown()
+
+	singersTeam, err := th.App.CreateTeam(&model.Team{
+		DisplayName: "Singers",
+		Name:        model.NewId(),
+		Email:       "singers@test.com",
+		Type:        model.TEAM_OPEN,
+	})
+	if err != nil {
+		t.Errorf("test team not created: %s", err.Error())
+	}
+
+	nerdsTeam, err := th.App.CreateTeam(&model.Team{
+		DisplayName: "Nerds",
+		Name:        model.NewId(),
+		Email:       "nerds@test.com",
+		Type:        model.TEAM_INVITE,
+	})
+	if err != nil {
+		t.Errorf("test team not created: %s", err.Error())
+	}
+
+	practiceChannel, err := th.App.CreateChannel(&model.Channel{
+		TeamId:      singersTeam.Id,
+		DisplayName: "Practices",
+		Name:        model.NewId(),
+		Type:        model.CHANNEL_OPEN,
+	}, false)
+	if err != nil {
+		t.Errorf("test channel not created: %s", err.Error())
+	}
+
+	experimentsChannel, err := th.App.CreateChannel(&model.Channel{
+		TeamId:      singersTeam.Id,
+		DisplayName: "Experiments",
+		Name:        model.NewId(),
+		Type:        model.CHANNEL_PRIVATE,
+	}, false)
+	if err != nil {
+		t.Errorf("test channel not created: %s", err.Error())
+	}
+
+	gleeGroup, err := th.App.CreateGroup(&model.Group{
+		Name:        model.NewId(),
+		DisplayName: "Glee Club",
+		RemoteId:    model.NewId(),
+		Source:      model.GroupSourceLdap,
+	})
+	if err != nil {
+		t.Errorf("test group not created: %s", err.Error())
+	}
+
+	scienceGroup, err := th.App.CreateGroup(&model.Group{
+		Name:        model.NewId(),
+		DisplayName: "Science Club",
+		RemoteId:    model.NewId(),
+		Source:      model.GroupSourceLdap,
+	})
+	if err != nil {
+		t.Errorf("test group not created: %s", err.Error())
+	}
+
+	_, err = th.App.CreateGroupSyncable(&model.GroupSyncable{
+		CanLeave:   true,
+		AutoAdd:    true,
+		GroupId:    gleeGroup.Id,
+		SyncableId: practiceChannel.Id,
+		Type:       model.GroupSyncableTypeChannel,
+	})
+	if err != nil {
+		t.Errorf("test groupchannel not created: %s", err.Error())
+	}
+
+	scienceTeamGroupSyncable, err := th.App.CreateGroupSyncable(&model.GroupSyncable{
+		CanLeave:   true,
+		AutoAdd:    false,
+		GroupId:    scienceGroup.Id,
+		SyncableId: nerdsTeam.Id,
+		Type:       model.GroupSyncableTypeTeam,
+	})
+	if err != nil {
+		t.Errorf("test groupteam not created: %s", err.Error())
+	}
+
+	scienceChannelGroupSyncable, err := th.App.CreateGroupSyncable(&model.GroupSyncable{
+		CanLeave:   true,
+		AutoAdd:    false,
+		GroupId:    scienceGroup.Id,
+		SyncableId: experimentsChannel.Id,
+		Type:       model.GroupSyncableTypeChannel,
+	})
+	if err != nil {
+		t.Errorf("test groupchannel not created: %s", err.Error())
+	}
+
+	singer1 := th.BasicUser
+	scientist1 := th.BasicUser2
+
+	_, err = th.App.CreateOrRestoreGroupMember(gleeGroup.Id, singer1.Id)
+	if err != nil {
+		t.Errorf("test groupmember not created: %s", err.Error())
+	}
+
+	scientistGroupMember, err := th.App.CreateOrRestoreGroupMember(scienceGroup.Id, scientist1.Id)
+	if err != nil {
+		t.Errorf("test groupmember not created: %s", err.Error())
+	}
+
+	pErr := th.App.PopulateSyncablesSince(0)
+	if pErr != nil {
+		t.Errorf("faild to populate syncables: %s", pErr.Error())
+	}
+
+	// Singer should be in team and channel
+	_, err = th.App.GetTeamMember(singersTeam.Id, singer1.Id)
+	if err != nil {
+		t.Errorf("error retrieving team member: %s", err.Error())
+	}
+	_, err = th.App.GetChannelMember(practiceChannel.Id, singer1.Id)
+	if err != nil {
+		t.Errorf("error retrieving channel member: %s", err.Error())
+	}
+
+	tMembers, err := th.App.GetTeamMembers(singersTeam.Id, 0, 999)
+	if err != nil {
+		t.Errorf("error retrieving team members: %s", err.Error())
+	}
+	expected := 1
+	actual := len(tMembers)
+	if actual != expected {
+		t.Errorf("expected %d team members but got %d", expected, actual)
+	}
+
+	cMembersCount, err := th.App.GetChannelMemberCount(practiceChannel.Id)
+	if err != nil {
+		t.Errorf("error retrieving team members: %s", err.Error())
+	}
+	if cMembersCount != int64(expected) {
+		t.Errorf("expected %d team member but got %d", expected, cMembersCount)
+	}
+
+	// Scientist should not be in team or channel
+	_, err = th.App.GetTeamMember(nerdsTeam.Id, scientist1.Id)
+	if err.Id != "store.sql_team.get_member.missing.app_error" {
+		t.Errorf("wrong error: %s", err.Id)
+	}
+
+	_, err = th.App.GetChannelMember(experimentsChannel.Id, scientist1.Id)
+	if err.Id != "store.sql_channel.get_member.missing.app_error" {
+		t.Errorf("wrong error: %s", err.Id)
+	}
+
+	tMembers, err = th.App.GetTeamMembers(nerdsTeam.Id, 0, 999)
+	if err != nil {
+		t.Errorf("error retrieving team members: %s", err.Error())
+	}
+	expected = 0
+	actual = len(tMembers)
+	if actual != expected {
+		t.Errorf("expected %d team members but got %d", expected, actual)
+	}
+
+	cMembersCount, err = th.App.GetChannelMemberCount(experimentsChannel.Id)
+	if err != nil {
+		t.Errorf("error retrieving team members: %s", err.Error())
+	}
+	if cMembersCount != int64(expected) {
+		t.Errorf("expected %d team members but got %d", expected, cMembersCount)
+	}
+
+	// update AutoAdd to true
+	scienceTeamGroupSyncable.AutoAdd = true
+	scienceTeamGroupSyncable, err = th.App.UpdateGroupSyncable(scienceTeamGroupSyncable)
+	if err != nil {
+		t.Errorf("error updating group syncable: %s", err.Error())
+	}
+
+	// Sync everything after syncable was created (proving that team updates trigger re-sync)
+	pErr = th.App.PopulateSyncablesSince(scientistGroupMember.CreateAt + 1)
+	if pErr != nil {
+		t.Errorf("faild to populate syncables: %s", pErr.Error())
+	}
+
+	// Scientist should be in team but not the channel
+	_, err = th.App.GetTeamMember(nerdsTeam.Id, scientist1.Id)
+	if err != nil {
+		t.Errorf("error retrieving team member: %s", err.Error())
+	}
+
+	_, err = th.App.GetChannelMember(experimentsChannel.Id, scientist1.Id)
+	if err.Id != "store.sql_channel.get_member.missing.app_error" {
+		t.Errorf("wrong error: %s", err.Id)
+	}
+
+	tMembers, err = th.App.GetTeamMembers(nerdsTeam.Id, 0, 999)
+	if err != nil {
+		t.Errorf("error retrieving team members: %s", err.Error())
+	}
+	expected = 1
+	actual = len(tMembers)
+	if actual != expected {
+		t.Errorf("expected %d team members but got %d", expected, actual)
+	}
+
+	expected = 0
+	cMembersCount, err = th.App.GetChannelMemberCount(experimentsChannel.Id)
+	if err != nil {
+		t.Errorf("error retrieving team members: %s", err.Error())
+	}
+	if cMembersCount != int64(expected) {
+		t.Errorf("expected %d team members but got %d", expected, cMembersCount)
+	}
+
+	// Update the channel syncable
+	scienceChannelGroupSyncable.AutoAdd = true
+	scienceChannelGroupSyncable, err = th.App.UpdateGroupSyncable(scienceChannelGroupSyncable)
+	if err != nil {
+		t.Errorf("error updating group syncable: %s", err.Error())
+	}
+
+	// Sync everything after syncable was created (proving that channel updates trigger re-sync)
+	pErr = th.App.PopulateSyncablesSince(scientistGroupMember.CreateAt + 1)
+	if pErr != nil {
+		t.Errorf("faild to populate syncables: %s", pErr.Error())
+	}
+
+	expected = 1
+	cMembersCount, err = th.App.GetChannelMemberCount(experimentsChannel.Id)
+	if err != nil {
+		t.Errorf("error retrieving team members: %s", err.Error())
+	}
+	if cMembersCount != int64(expected) {
+		t.Errorf("expected %d team members but got %d", expected, cMembersCount)
+	}
+
+	// singer leaves team and channel
+	err = th.App.LeaveChannel(practiceChannel.Id, singer1.Id)
+	if err != nil {
+		t.Errorf("error leaving channel: %s", err.Error())
+	}
+	err = th.App.LeaveTeam(singersTeam, singer1, "")
+	if err != nil {
+		t.Errorf("error leaving team: %s", err.Error())
+	}
+
+	// Even re-syncing from the beginning doesn't re-add to channel or team
+	pErr = th.App.PopulateSyncablesSince(0)
+	if pErr != nil {
+		t.Errorf("faild to populate syncables: %s", pErr.Error())
+	}
+
+	// Singer should not be in team or channel
+	tMember, err := th.App.GetTeamMember(singersTeam.Id, singer1.Id)
+	if err != nil {
+		t.Errorf("error retrieving team member: %s", err.Error())
+	}
+	if tMember.DeleteAt == 0 {
+		t.Error("expected team member to remain deleted")
+	}
+
+	_, err = th.App.GetChannelMember(practiceChannel.Id, singer1.Id)
+	if err == nil {
+		t.Error("Expected channel member to remain deleted")
+	}
+
+	// Ensure members are in channel
+	_, err = th.App.AddChannelMember(scientist1.Id, experimentsChannel, "", "", false)
+	if err != nil {
+		t.Errorf("unable to add user to channel: %s", err.Error())
+	}
+
+	// Add other user so that user can leave channel
+	_, err = th.App.AddTeamMember(singersTeam.Id, singer1.Id)
+	if err != nil {
+		t.Errorf("unable to add user to team: %s", err.Error())
+	}
+	_, err = th.App.AddChannelMember(singer1.Id, experimentsChannel, "", "", false)
+	if err != nil {
+		t.Errorf("unable to add user to channel: %s", err.Error())
+	}
+
+	// the channel syncable is updated
+	scienceChannelGroupSyncable.CanLeave = false
+	scienceChannelGroupSyncable, err = th.App.UpdateGroupSyncable(scienceChannelGroupSyncable)
+	if err != nil {
+		t.Errorf("error updating group syncable: %s", err.Error())
+	}
+
+	pErr = th.App.PopulateSyncablesSince(0)
+	if pErr != nil {
+		t.Errorf("faild to populate syncables: %s", pErr.Error())
+	}
+
+	timeBeforeLeaving := model.GetMillis()
+
+	// User leaves channel
+	err = th.App.LeaveChannel(experimentsChannel.Id, scientist1.Id)
+	if err != nil {
+		t.Errorf("unable to add user to channel: %s", err.Error())
+	}
+
+	timeAfterLeaving := model.GetMillis()
+
+	// Purging channelmemberhistory doesn't re-add user to channel
+	result := <-th.App.Srv.Store.ChannelMemberHistory().PermanentDeleteBatch(timeBeforeLeaving, 1000)
+	if result.Err != nil {
+		t.Errorf("error permanently deleting channelmemberhistory: %s", result.Err.Error())
+	}
+
+	pErr = th.App.PopulateSyncablesSince(scienceChannelGroupSyncable.UpdateAt)
+	if pErr != nil {
+		t.Errorf("failed to populate syncables: %s", pErr.Error())
+	}
+
+	_, err = th.App.GetChannelMember(experimentsChannel.Id, scientist1.Id)
+	if err == nil {
+		t.Error("Expected channel member to remain deleted")
+	}
+
+	// Purging channelmemberhistory doesn't re-add user to channel
+	result = <-th.App.Srv.Jobs.Store.ChannelMemberHistory().PermanentDeleteBatch(timeAfterLeaving, 1000)
+	if result.Err != nil {
+		t.Errorf("error permanently deleting channelmemberhistory: %s", result.Err.Error())
+	}
+
+	pErr = th.App.PopulateSyncablesSince(scienceChannelGroupSyncable.UpdateAt)
+	if pErr != nil {
+		t.Errorf("failed to populate syncables: %s", pErr.Error())
+	}
+
+	// Channel member is re-added.
+	_, err = th.App.GetChannelMember(experimentsChannel.Id, scientist1.Id)
+	if err != nil {
+		t.Errorf("expected channel member: %s", err.Error())
+	}
+}