[MM 12464] Include DM/GM Channels and Their Posts in the Bulk Export (#10421)

* transplant the existing PR into the working tree

* start addressing review comments

* move existing direct channel export code into this branch

* modify channel exporter to use squirell and populate members in two steps

* use squirrel to build sql queries for channel and dm/gm export methods

* remove debug helpers and use Username instead of UserId

* unit test for DM Channel exporter

* add more unit tests for channel export

* add test for DM/GM post export

* checkpoint with failing test for postgres

* use getQueryBuilder to make sure squirrel uses the correct formatting
for each database

* add a test for post export

* fix shadowed vars that broke the build

* address review comments and add tests to support it

* address review comments and add a mlog call

* s/Info/Debug/

* address review comments in post_store

* address review comments in channel_store

* address review comments in export

* address review comment in post_store: drop GroupBy

* address review comment on supplier: move getQueryBuilder to sqlstore

* address review comments: explicit TearDown

* address review comments: improve test coverage

* address review comments: make sure public and private channels are excluded

* address review comments: improve test coverage

* address review comments: make sure Channels table gets truncated after
each test

* more cleanups and better assertions

* wrap PostStore in a StoreTestWithSqlSupplier

* last minute changes: improve post export test coverage and check members

* address review comments: make sure all posts have their channel
members set

* address review comments: make sure all posts have their ChannelMembers
exported correctly

* gofmt fix

* sort channels so it's possible to assert on index
Этот коммит содержится в:
Fernando Correa Neto
2019-03-15 12:28:43 -03:00
коммит произвёл Jesús Espino
родитель eb49713c96
Коммит 9abd4dd7dc
17 изменённых файлов: 925 добавлений и 22 удалений

Просмотреть файл

@@ -15,6 +15,7 @@ import (
"sync/atomic"
"time"
sq "github.com/Masterminds/squirrel"
"github.com/dyatlov/go-opengraph/opengraph"
"github.com/go-sql-driver/mysql"
"github.com/lib/pq"
@@ -1051,6 +1052,14 @@ func (ss *SqlSupplier) DropAllTables() {
ss.master.TruncateTables()
}
func (ss *SqlSupplier) getQueryBuilder() sq.StatementBuilderType {
builder := sq.StatementBuilder.PlaceholderFormat(sq.Question)
if ss.DriverName() == model.DATABASE_DRIVER_POSTGRES {
builder = builder.PlaceholderFormat(sq.Dollar)
}
return builder
}
type mattermConverter struct{}
func (me mattermConverter) ToDb(val interface{}) (interface{}, error) {