MM-9072/MM-10185 Force-convert the encoding of OpenGraph metadata to UTF-8 (#8631)

* Force-convert non-UTF8 HTML to UTF8 before opengraph processing

* Split the force-encoding function

* Add benchmark Test for the forceHTMLEncodingToUTF8()

```
Running tool: /home/comet/go-v1.9.2/bin/go test -benchmem -run=^$ github.com/mattermost/mattermost-server/app -bench ^BenchmarkForceHTMLEncodingToUTF8$

[03:32:58 KST 2018/04/21] [INFO] (github.com/mattermost/mattermost-server/app.TestMain:28) -test.run used, not creating temporary containers
goos: linux
goarch: amd64
pkg: github.com/mattermost/mattermost-server/app
BenchmarkForceHTMLEncodingToUTF8/with_converting-4         	  100000	     11201 ns/op	   18704 B/op	      32 allocs/op
BenchmarkForceHTMLEncodingToUTF8/without_converting-4      	  300000	      3931 ns/op	    4632 B/op	      13 allocs/op
PASS
ok  	github.com/mattermost/mattermost-server/app	2.703s
Success: Benchmarks passed.
```

* Remove an unnecessary constraint

* Add pre-check if content-type header is already utf-8

* Move the checking for utf-8 into forceHTMLEncodingToUTF8() for testing

* Revert df3f347213faa0d023c26d201fa6531f46391086..HEAD, without Gopkg.lock
Этот коммит содержится в:
Hyeseong Kim
2018-05-01 01:54:11 +09:00
коммит произвёл Christopher Speller
родитель 9c5815ee41
Коммит e73f1d7314
38 изменённых файлов: 157135 добавлений и 2 удалений

15
Gopkg.lock сгенерированный
Просмотреть файл

@@ -562,6 +562,7 @@
"bpf",
"html",
"html/atom",
"html/charset",
"idna",
"internal/iana",
"internal/socket",
@@ -585,12 +586,24 @@
packages = [
"collate",
"collate/build",
"encoding",
"encoding/charmap",
"encoding/htmlindex",
"encoding/internal",
"encoding/internal/identifier",
"encoding/japanese",
"encoding/korean",
"encoding/simplifiedchinese",
"encoding/traditionalchinese",
"encoding/unicode",
"internal/colltab",
"internal/gen",
"internal/tag",
"internal/triegen",
"internal/ucd",
"internal/utf8internal",
"language",
"runes",
"secure/bidirule",
"transform",
"unicode/bidi",
@@ -659,6 +672,6 @@
[solve-meta]
analyzer-name = "dep"
analyzer-version = 1
inputs-digest = "4f92b22eac4e8efa0aa098a6b1f1599ab087160296bd5fb74a9fad5366bbe1bc"
inputs-digest = "c9e6f408ca532c6d09084fcd86678986225e8810585cbc0476a586f1d1969066"
solver-name = "gps-cdcl"
solver-version = 1