MM-51486: Implementing UTS 46 to check for preview blocklist (#22601)

There is a standard IDNA2008 which defines how unicode characters
in domain names can be compared using UTS (Unicode Technical Standard) 46.

We use that to convert any links in messages to ASCII and
then compare with the domain list in config.

https://mattermost.atlassian.net/browse/MM-51486

```release-note
NONE
```
Этот коммит содержится в:
Agniva De Sarker
2023-03-23 13:03:59 +05:30
коммит произвёл GitHub
родитель c943ed6859
Коммит 19c7c6fa7a
3 изменённых файлов: 46 добавлений и 8 удалений

Просмотреть файл

@@ -1664,6 +1664,21 @@ func TestGetFirstLinkAndImages(t *testing.T) {
ExpectedFirstLink: "",
ExpectedImages: []string{},
},
"test with denied domain as username, password": {
Input: "link as http://example.com:example.com@example1.com/link1",
ExpectedFirstLink: "http://example.com:example.com@example1.com/link1",
ExpectedImages: []string{},
},
"test with URL encoded": {
Input: "link as https://example%E3%80%82com/link1",
ExpectedFirstLink: "",
ExpectedImages: []string{},
},
"test with unicode": {
Input: "link as https://example。com/link1",
ExpectedFirstLink: "",
ExpectedImages: []string{},
},
} {
th.App.UpdateConfig(func(cfg *model.Config) {
*cfg.ServiceSettings.RestrictLinkPreviews = "example.com, test.com"