Files
mostlymatter/api4/file.go
Lev fae4f60b33 MM-7633: Optimize memory utilization during file uploads (#9835)
* MM-7633: Optimize memory utilization during file uploads

Refactored the file upload code to reduce redundant buffering and stream
directly to the file store. Added tests.

Benchmark results:
```
levs-mbp:mattermost-server levb$  go test -v -run nothing -bench Upload -benchmem ./app
...
BenchmarkUploadFile/random-5Mb-gif-raw-ish_DoUploadFile-4         	      10	 122598031 ns/op	21211370 B/op	    1008 allocs/op
BenchmarkUploadFile/random-5Mb-gif-raw_UploadFileTask-4           	     100	  20211926 ns/op	 5678750 B/op	     126 allocs/op
BenchmarkUploadFile/random-5Mb-gif-UploadFiles-4                  	       2	1037051184 ns/op	81806360 B/op	 3705013 allocs/op
BenchmarkUploadFile/random-5Mb-gif-UploadFileTask-4               	       2	 933644431 ns/op	67015868 B/op	 3704410 allocs/op
BenchmarkUploadFile/random-2Mb-jpg-raw-ish_DoUploadFile-4         	     100	  13110509 ns/op	 6032614 B/op	    8052 allocs/op
BenchmarkUploadFile/random-2Mb-jpg-raw_UploadFileTask-4           	     100	  10729867 ns/op	 1738303 B/op	     125 allocs/op
BenchmarkUploadFile/random-2Mb-jpg-UploadFiles-4                  	       2	 925274912 ns/op	70326352 B/op	 3718856 allocs/op
BenchmarkUploadFile/random-2Mb-jpg-UploadFileTask-4               	       2	 995033336 ns/op	58113796 B/op	 3710943 allocs/op
BenchmarkUploadFile/zero-10Mb-raw-ish_DoUploadFile-4              	      30	  50777211 ns/op	54791929 B/op	    2714 allocs/op
BenchmarkUploadFile/zero-10Mb-raw_UploadFileTask-4                	      50	  36387339 ns/op	10503920 B/op	     126 allocs/op
BenchmarkUploadFile/zero-10Mb-UploadFiles-4                       	      30	  48657678 ns/op	54791948 B/op	    2719 allocs/op
BenchmarkUploadFile/zero-10Mb-UploadFileTask-4                    	      50	  37506467 ns/op	31492060 B/op	     131 allocs/op
...
```

https://mattermost.atlassian.net/browse/MM-7633 https://github.com/mattermost/mattermost-server/issues/7801

[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]
- [x] Added or updated unit tests (required for all new features)
- [ ] Added API documentation (required for all new APIs)
- [ ] All new/modified APIs include changes to the drivers
  *N/A*???
- [x] Includes text changes and localization file ([.../i18n/en.json](https://github.com/mattermost/mattermost-server/blob/master/i18n/en.json)) updates

Overview of changes:
- api4
    - Replaced `uploadFile` handler with `uploadFileStream` that reduces
      unnecessary buffering.
    - Added/refactored tests for the new API.
    - Refactored apitestlib/Check...Status functions.
- app
    - Added App.UploadFileTask, a more efficient refactor of UploadFile.
    - Consistently set `FileInfo.HasPreviewImage`
    - Added benchmarks for the new and prior implementations
    - Replaced passing around `*image.Image` with `image.Image` in the
      existing code.
- model
    - Added a more capable `client4.UploadFiles` API to match the new server
      API’s capabilities.
- I18n
    - Replaced `api.file.upload_file.bad_parse.app_error` with a more generic
      `api.file.upload_file.read_request.app_error`
- plugin
    - Removed type `plugin.multiPluginHookRunnerFunc` in favor of using
      `func(hooks Hooks) bool` explicitly, to help with testing
- tests
    - Added test files for testing images

Still remaining, but can be separate PRs - please let me know the preferred
course of action
    - Investigate JS client API - how does it do multipart?
    - Performance loss from old code on (small) image processing?
    - Deprecate the old functions, change other API implementations to use
      UploadFileTask

Definitely separate future PRs - should I file tickets foe these?
    - Only invoke t.readAll() if there are indeed applicable plugins to run
    - Find a way to leverage goexif buffer rather than re-reading

Suggested long-term improvements - should I file separate tickets for these?
- Actually allow uploading of large (GB-sized) files. This may require a
  change in how the file is passed to plugins.
- (Many) api4 tests should probably be subtests and share a server setup -
  will be much faster
- Performance improvements in image processing (goexif/thumbnail/preview)
  (maybe use https://mattermost.atlassian.net/browse/MM-10188 for this)

Questions:
1. I am commiting MBs of test images, are there better alternatives? I can
   probably create much less dense images that would take up considerably less
   space, even at pretty large sizes
2. I18n: Do I need to do anything special for the string change? Or just wait
   until it gets picked up and translated/updated?
3. The image dimensions are flipped in resulting FileInfo to match the actual
   orientation. Is this by design? Should add a test for it, perhaps?
4. What to do in the case of partial success? S3 but not DB, some files but not
   others? For now, just doing what the old code did, I think.
5. Make maxUploadDrainBytes configurable? Also, should this be the systemic
   behavior of all APIs with non-empty body? Otherwise dropped altogether?
   Check all other ioutil.ReadAll() from sockets. Find a way to set a total
   byte limit on request Body?

* WIP - Fixed for GetPluginsEnvironment() changes

* WIP - PR feedback

1. Refactored checkHTTPStatus to improve failure messages
2. Use `var name []type` rather than `name := ([]type)(nil)`
3. Replaced single-letter `p` with a more intention-revealing `part`
4. Added tests for full image size, `HasPreviewImage`

* WIP - rebased (c.Session->c.App.Session)

* WIP - PR feedback: eliminated use of Request.MultipartReader

Instead of hacking the request object to use r.MultipartReader now have own
functions `parseMultipartRequestHeader` and `multipartReader` eliminating the
need to hack the request object to use Request.MultipartReader limitations.

* WIP - PR feedback: UploadFileX with functional options

* WIP - PR feedback: style

* WIP - PR feedback: errors cleanup

* WIP - clarified File Upload benchmarks

* WIP - PR feedback: display the value of erroneous formname

* WIP - PR feedback: fixed handling of multiple channel_ids

* WIP - rebased from master - fixed tests

* PR Feedback

* PR feedback - moved client4.UploadFiles to _test for now
2018-12-13 22:32:07 +01:00

791 строка
21 KiB
Go

// Copyright (c) 2017-present Mattermost, Inc. All Rights Reserved.
// See License.txt for license information.
package api4
import (
"bytes"
"crypto/subtle"
"io"
"io/ioutil"
"mime"
"mime/multipart"
"net/http"
"net/url"
"strconv"
"strings"
"time"
"github.com/mattermost/mattermost-server/app"
"github.com/mattermost/mattermost-server/model"
"github.com/mattermost/mattermost-server/utils"
)
const (
FILE_TEAM_ID = "noteam"
PREVIEW_IMAGE_TYPE = "image/jpeg"
THUMBNAIL_IMAGE_TYPE = "image/jpeg"
)
var UNSAFE_CONTENT_TYPES = [...]string{
"application/javascript",
"application/ecmascript",
"text/javascript",
"text/ecmascript",
"application/x-javascript",
"text/html",
}
var MEDIA_CONTENT_TYPES = [...]string{
"image/jpeg",
"image/png",
"image/bmp",
"image/gif",
"video/avi",
"video/mpeg",
"video/mp4",
"audio/mpeg",
"audio/wav",
}
const maxUploadDrainBytes = (10 * 1024 * 1024) // 10Mb
const maxMultipartFormDataBytes = 10 * 1024 // 10Kb
func (api *API) InitFile() {
api.BaseRoutes.Files.Handle("", api.ApiSessionRequired(uploadFileStream)).Methods("POST")
api.BaseRoutes.File.Handle("", api.ApiSessionRequiredTrustRequester(getFile)).Methods("GET")
api.BaseRoutes.File.Handle("/thumbnail", api.ApiSessionRequiredTrustRequester(getFileThumbnail)).Methods("GET")
api.BaseRoutes.File.Handle("/link", api.ApiSessionRequired(getFileLink)).Methods("GET")
api.BaseRoutes.File.Handle("/preview", api.ApiSessionRequiredTrustRequester(getFilePreview)).Methods("GET")
api.BaseRoutes.File.Handle("/info", api.ApiSessionRequired(getFileInfo)).Methods("GET")
api.BaseRoutes.PublicFile.Handle("", api.ApiHandler(getPublicFile)).Methods("GET")
}
func uploadFile(c *Context, w http.ResponseWriter, r *http.Request) {
defer io.Copy(ioutil.Discard, r.Body)
if !*c.App.Config().FileSettings.EnableFileAttachments {
c.Err = model.NewAppError("uploadFile", "api.file.attachments.disabled.app_error", nil, "", http.StatusNotImplemented)
return
}
if r.ContentLength > *c.App.Config().FileSettings.MaxFileSize {
c.Err = model.NewAppError("uploadFile", "api.file.upload_file.too_large.app_error", nil, "", http.StatusRequestEntityTooLarge)
return
}
now := time.Now()
var resStruct *model.FileUploadResponse
var appErr *model.AppError
if err := r.ParseMultipartForm(*c.App.Config().FileSettings.MaxFileSize); err != nil && err != http.ErrNotMultipart {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
} else if err == http.ErrNotMultipart {
defer r.Body.Close()
c.RequireChannelId()
c.RequireFilename()
if c.Err != nil {
return
}
channelId := c.Params.ChannelId
filename := c.Params.Filename
if !c.App.SessionHasPermissionToChannel(c.App.Session, channelId, model.PERMISSION_UPLOAD_FILE) {
c.SetPermissionError(model.PERMISSION_UPLOAD_FILE)
return
}
resStruct, appErr = c.App.UploadFiles(
FILE_TEAM_ID,
channelId,
c.App.Session.UserId,
[]io.ReadCloser{r.Body},
[]string{filename},
[]string{},
now,
)
} else {
m := r.MultipartForm
props := m.Value
if len(props["channel_id"]) == 0 {
c.SetInvalidParam("channel_id")
return
}
channelId := props["channel_id"][0]
c.Params.ChannelId = channelId
c.RequireChannelId()
if c.Err != nil {
return
}
if !c.App.SessionHasPermissionToChannel(c.App.Session, channelId, model.PERMISSION_UPLOAD_FILE) {
c.SetPermissionError(model.PERMISSION_UPLOAD_FILE)
return
}
resStruct, appErr = c.App.UploadMultipartFiles(
FILE_TEAM_ID,
channelId,
c.App.Session.UserId,
m.File["files"],
m.Value["client_ids"],
now,
)
}
if appErr != nil {
c.Err = appErr
return
}
w.WriteHeader(http.StatusCreated)
w.Write([]byte(resStruct.ToJson()))
}
func parseMultipartRequestHeader(req *http.Request) (boundary string, err error) {
v := req.Header.Get("Content-Type")
if v == "" {
return "", http.ErrNotMultipart
}
d, params, err := mime.ParseMediaType(v)
if err != nil || d != "multipart/form-data" {
return "", http.ErrNotMultipart
}
boundary, ok := params["boundary"]
if !ok {
return "", http.ErrMissingBoundary
}
return boundary, nil
}
func multipartReader(req *http.Request, stream io.Reader) (*multipart.Reader, error) {
boundary, err := parseMultipartRequestHeader(req)
if err != nil {
return nil, err
}
if stream != nil {
return multipart.NewReader(stream, boundary), nil
} else {
return multipart.NewReader(req.Body, boundary), nil
}
}
func uploadFileStream(c *Context, w http.ResponseWriter, r *http.Request) {
// Drain any remaining bytes in the request body, up to a limit
defer io.CopyN(ioutil.Discard, r.Body, maxUploadDrainBytes)
if !*c.App.Config().FileSettings.EnableFileAttachments {
c.Err = model.NewAppError("uploadFileStream",
"api.file.attachments.disabled.app_error",
nil, "", http.StatusNotImplemented)
return
}
// Parse the post as a regular form (in practice, use the URL values
// since we never expect a real application/x-www-form-urlencoded
// form).
if r.Form == nil {
err := r.ParseForm()
if err != nil {
c.Err = model.NewAppError("uploadFileStream",
"api.file.upload_file.read_request.app_error",
nil, err.Error(), http.StatusBadRequest)
return
}
}
timestamp := time.Now()
var fileUploadResponse *model.FileUploadResponse
_, err := parseMultipartRequestHeader(r)
switch err {
case nil:
fileUploadResponse = uploadFileMultipart(c, r, nil, timestamp)
case http.ErrNotMultipart:
fileUploadResponse = uploadFileSimple(c, r, timestamp)
default:
c.Err = model.NewAppError("uploadFileStream",
"api.file.upload_file.read_request.app_error",
nil, err.Error(), http.StatusBadRequest)
}
if c.Err != nil {
return
}
// Write the response values to the output upon return
w.WriteHeader(http.StatusCreated)
w.Write([]byte(fileUploadResponse.ToJson()))
}
// uploadFileSimple uploads a file from a simple POST with the file in the request body
func uploadFileSimple(c *Context, r *http.Request, timestamp time.Time) *model.FileUploadResponse {
// Simple POST with the file in the body and all metadata in the args.
c.RequireChannelId()
c.RequireFilename()
if c.Err != nil {
return nil
}
if !c.App.SessionHasPermissionToChannel(c.App.Session, c.Params.ChannelId, model.PERMISSION_UPLOAD_FILE) {
c.SetPermissionError(model.PERMISSION_UPLOAD_FILE)
return nil
}
clientId := r.Form.Get("client_id")
info, appErr := c.App.UploadFileX(c.Params.ChannelId, c.Params.Filename, r.Body,
app.UploadFileSetTeamId(FILE_TEAM_ID),
app.UploadFileSetUserId(c.App.Session.UserId),
app.UploadFileSetTimestamp(timestamp),
app.UploadFileSetContentLength(r.ContentLength),
app.UploadFileSetClientId(clientId))
if appErr != nil {
c.Err = appErr
return nil
}
fileUploadResponse := &model.FileUploadResponse{
FileInfos: []*model.FileInfo{info},
}
if clientId != "" {
fileUploadResponse.ClientIds = []string{clientId}
}
return fileUploadResponse
}
// uploadFileMultipart parses and uploads file(s) from a mime/multipart
// request. It pre-buffers up to the first part which is either the (a)
// `channel_id` value, or (b) a file. Then in case of (a) it re-processes the
// entire message recursively calling itself in stream mode. In case of (b) it
// calls to uploadFileMultipartLegacy for legacy support
func uploadFileMultipart(c *Context, r *http.Request, asStream io.Reader, timestamp time.Time) *model.FileUploadResponse {
expectClientIds := true
var clientIds []string
resp := model.FileUploadResponse{
FileInfos: []*model.FileInfo{},
ClientIds: []string{},
}
var buf *bytes.Buffer
var mr *multipart.Reader
var err error
if asStream == nil {
// We need to buffer until we get the channel_id, or the first file.
buf = &bytes.Buffer{}
mr, err = multipartReader(r, io.TeeReader(r.Body, buf))
} else {
mr, err = multipartReader(r, asStream)
}
if err != nil {
c.Err = model.NewAppError("uploadFileMultipart",
"api.file.upload_file.read_request.app_error",
nil, err.Error(), http.StatusBadRequest)
return nil
}
nFiles := 0
NEXT_PART:
for {
part, err := mr.NextPart()
if err == io.EOF {
break
}
if err != nil {
c.Err = model.NewAppError("uploadFileMultipart",
"api.file.upload_file.read_request.app_error",
nil, err.Error(), http.StatusBadRequest)
return nil
}
// Parse any form fields in the multipart.
formname := part.FormName()
if formname == "" {
continue
}
filename := part.FileName()
if filename == "" {
var b bytes.Buffer
_, err := io.CopyN(&b, part, maxMultipartFormDataBytes)
if err != nil && err != io.EOF {
c.Err = model.NewAppError("uploadFileMultipart",
"api.file.upload_file.read_form_value.app_error",
map[string]interface{}{"Formname": formname},
err.Error(), http.StatusBadRequest)
return nil
}
v := b.String()
switch formname {
case "channel_id":
if c.Params.ChannelId != "" && c.Params.ChannelId != v {
c.Err = model.NewAppError("uploadFileMultipart",
"api.file.upload_file.multiple_channel_ids.app_error",
nil, "", http.StatusBadRequest)
return nil
}
if v != "" {
c.Params.ChannelId = v
}
// Got channel_id, re-process the entire post
// in the streaming mode.
if asStream == nil {
return uploadFileMultipart(c, r, io.MultiReader(buf, r.Body), timestamp)
}
case "client_ids":
if !expectClientIds {
c.SetInvalidParam("client_ids")
return nil
}
clientIds = append(clientIds, v)
default:
c.SetInvalidParam(formname)
return nil
}
continue NEXT_PART
}
// A file part.
if c.Params.ChannelId == "" && asStream == nil {
// Got file before channel_id, fall back to legacy buffered mode
mr, err = multipartReader(r, io.MultiReader(buf, r.Body))
if err != nil {
c.Err = model.NewAppError("uploadFileMultipart",
"api.file.upload_file.read_request.app_error",
nil, err.Error(), http.StatusBadRequest)
return nil
}
return uploadFileMultipartLegacy(c, mr, timestamp)
}
c.RequireChannelId()
if c.Err != nil {
return nil
}
if !c.App.SessionHasPermissionToChannel(c.App.Session, c.Params.ChannelId, model.PERMISSION_UPLOAD_FILE) {
c.SetPermissionError(model.PERMISSION_UPLOAD_FILE)
return nil
}
// If there's no clientIds when the first file comes, expect
// none later.
if nFiles == 0 && len(clientIds) == 0 {
expectClientIds = false
}
// Must have a exactly one client ID for each file.
clientId := ""
if expectClientIds {
if nFiles >= len(clientIds) {
c.SetInvalidParam("client_ids")
return nil
}
clientId = clientIds[nFiles]
}
info, appErr := c.App.UploadFileX(c.Params.ChannelId, filename, part,
app.UploadFileSetTeamId(FILE_TEAM_ID),
app.UploadFileSetUserId(c.App.Session.UserId),
app.UploadFileSetTimestamp(timestamp),
app.UploadFileSetContentLength(-1),
app.UploadFileSetClientId(clientId))
if appErr != nil {
c.Err = appErr
return nil
}
// add to the response
resp.FileInfos = append(resp.FileInfos, info)
if expectClientIds {
resp.ClientIds = append(resp.ClientIds, clientId)
}
nFiles++
}
// Verify that the number of ClientIds matched the number of files.
if expectClientIds && len(clientIds) != nFiles {
c.Err = model.NewAppError("uploadFileMultipart",
"api.file.upload_file.incorrect_number_of_client_ids.app_error",
map[string]interface{}{"NumClientIds": len(clientIds), "NumFiles": nFiles},
"", http.StatusBadRequest)
return nil
}
return &resp
}
// uploadFileMultipartLegacy reads, buffers, and then uploads the message,
// borrowing from http.ParseMultipartForm. If successful it returns a
// *model.FileUploadResponse filled in with the individual model.FileInfo's.
func uploadFileMultipartLegacy(c *Context, mr *multipart.Reader,
timestamp time.Time) *model.FileUploadResponse {
// Parse the entire form.
form, err := mr.ReadForm(*c.App.Config().FileSettings.MaxFileSize)
if err != nil {
c.Err = model.NewAppError("uploadFileMultipartLegacy",
"api.file.upload_file.read_request.app_error",
nil, err.Error(), http.StatusInternalServerError)
return nil
}
// get and validate the channel Id, permission to upload there.
if len(form.Value["channel_id"]) == 0 {
c.SetInvalidParam("channel_id")
return nil
}
channelId := form.Value["channel_id"][0]
c.Params.ChannelId = channelId
c.RequireChannelId()
if c.Err != nil {
return nil
}
if !c.App.SessionHasPermissionToChannel(c.App.Session, channelId, model.PERMISSION_UPLOAD_FILE) {
c.SetPermissionError(model.PERMISSION_UPLOAD_FILE)
return nil
}
// Check that we have either no client IDs, or one per file.
clientIds := form.Value["client_ids"]
fileHeaders := form.File["files"]
if len(clientIds) != 0 && len(clientIds) != len(fileHeaders) {
c.Err = model.NewAppError("uploadFilesMultipartBuffered",
"api.file.upload_file.incorrect_number_of_client_ids.app_error",
map[string]interface{}{"NumClientIds": len(clientIds), "NumFiles": len(fileHeaders)},
"", http.StatusBadRequest)
return nil
}
resp := model.FileUploadResponse{
FileInfos: []*model.FileInfo{},
ClientIds: []string{},
}
for i, fileHeader := range fileHeaders {
f, err := fileHeader.Open()
if err != nil {
c.Err = model.NewAppError("uploadFileMultipartLegacy",
"api.file.upload_file.read_request.app_error",
nil, err.Error(), http.StatusBadRequest)
return nil
}
clientId := ""
if len(clientIds) > 0 {
clientId = clientIds[i]
}
info, appErr := c.App.UploadFileX(c.Params.ChannelId, fileHeader.Filename, f,
app.UploadFileSetTeamId(FILE_TEAM_ID),
app.UploadFileSetUserId(c.App.Session.UserId),
app.UploadFileSetTimestamp(timestamp),
app.UploadFileSetContentLength(-1),
app.UploadFileSetClientId(clientId))
f.Close()
if appErr != nil {
c.Err = appErr
return nil
}
resp.FileInfos = append(resp.FileInfos, info)
if clientId != "" {
resp.ClientIds = append(resp.ClientIds, clientId)
}
}
return &resp
}
func getFile(c *Context, w http.ResponseWriter, r *http.Request) {
c.RequireFileId()
if c.Err != nil {
return
}
forceDownload, convErr := strconv.ParseBool(r.URL.Query().Get("download"))
if convErr != nil {
forceDownload = false
}
info, err := c.App.GetFileInfo(c.Params.FileId)
if err != nil {
c.Err = err
return
}
if info.CreatorId != c.App.Session.UserId && !c.App.SessionHasPermissionToChannelByPost(c.App.Session, info.PostId, model.PERMISSION_READ_CHANNEL) {
c.SetPermissionError(model.PERMISSION_READ_CHANNEL)
return
}
fileReader, err := c.App.FileReader(info.Path)
if err != nil {
c.Err = err
c.Err.StatusCode = http.StatusNotFound
return
}
defer fileReader.Close()
err = writeFileResponse(info.Name, info.MimeType, info.Size, fileReader, forceDownload, w, r)
if err != nil {
c.Err = err
return
}
}
func getFileThumbnail(c *Context, w http.ResponseWriter, r *http.Request) {
c.RequireFileId()
if c.Err != nil {
return
}
forceDownload, convErr := strconv.ParseBool(r.URL.Query().Get("download"))
if convErr != nil {
forceDownload = false
}
info, err := c.App.GetFileInfo(c.Params.FileId)
if err != nil {
c.Err = err
return
}
if info.CreatorId != c.App.Session.UserId && !c.App.SessionHasPermissionToChannelByPost(c.App.Session, info.PostId, model.PERMISSION_READ_CHANNEL) {
c.SetPermissionError(model.PERMISSION_READ_CHANNEL)
return
}
if info.ThumbnailPath == "" {
c.Err = model.NewAppError("getFileThumbnail", "api.file.get_file_thumbnail.no_thumbnail.app_error", nil, "file_id="+info.Id, http.StatusBadRequest)
return
}
fileReader, err := c.App.FileReader(info.ThumbnailPath)
if err != nil {
c.Err = err
c.Err.StatusCode = http.StatusNotFound
return
}
defer fileReader.Close()
err = writeFileResponse(info.Name, THUMBNAIL_IMAGE_TYPE, 0, fileReader, forceDownload, w, r)
if err != nil {
c.Err = err
return
}
}
func getFileLink(c *Context, w http.ResponseWriter, r *http.Request) {
c.RequireFileId()
if c.Err != nil {
return
}
if !c.App.Config().FileSettings.EnablePublicLink {
c.Err = model.NewAppError("getPublicLink", "api.file.get_public_link.disabled.app_error", nil, "", http.StatusNotImplemented)
return
}
info, err := c.App.GetFileInfo(c.Params.FileId)
if err != nil {
c.Err = err
return
}
if info.CreatorId != c.App.Session.UserId && !c.App.SessionHasPermissionToChannelByPost(c.App.Session, info.PostId, model.PERMISSION_READ_CHANNEL) {
c.SetPermissionError(model.PERMISSION_READ_CHANNEL)
return
}
if len(info.PostId) == 0 {
c.Err = model.NewAppError("getPublicLink", "api.file.get_public_link.no_post.app_error", nil, "file_id="+info.Id, http.StatusBadRequest)
return
}
resp := make(map[string]string)
resp["link"] = c.App.GeneratePublicLink(c.GetSiteURLHeader(), info)
w.Write([]byte(model.MapToJson(resp)))
}
func getFilePreview(c *Context, w http.ResponseWriter, r *http.Request) {
c.RequireFileId()
if c.Err != nil {
return
}
forceDownload, convErr := strconv.ParseBool(r.URL.Query().Get("download"))
if convErr != nil {
forceDownload = false
}
info, err := c.App.GetFileInfo(c.Params.FileId)
if err != nil {
c.Err = err
return
}
if info.CreatorId != c.App.Session.UserId && !c.App.SessionHasPermissionToChannelByPost(c.App.Session, info.PostId, model.PERMISSION_READ_CHANNEL) {
c.SetPermissionError(model.PERMISSION_READ_CHANNEL)
return
}
if info.PreviewPath == "" {
c.Err = model.NewAppError("getFilePreview", "api.file.get_file_preview.no_preview.app_error", nil, "file_id="+info.Id, http.StatusBadRequest)
return
}
fileReader, err := c.App.FileReader(info.PreviewPath)
if err != nil {
c.Err = err
c.Err.StatusCode = http.StatusNotFound
return
}
defer fileReader.Close()
err = writeFileResponse(info.Name, PREVIEW_IMAGE_TYPE, 0, fileReader, forceDownload, w, r)
if err != nil {
c.Err = err
return
}
}
func getFileInfo(c *Context, w http.ResponseWriter, r *http.Request) {
c.RequireFileId()
if c.Err != nil {
return
}
info, err := c.App.GetFileInfo(c.Params.FileId)
if err != nil {
c.Err = err
return
}
if info.CreatorId != c.App.Session.UserId && !c.App.SessionHasPermissionToChannelByPost(c.App.Session, info.PostId, model.PERMISSION_READ_CHANNEL) {
c.SetPermissionError(model.PERMISSION_READ_CHANNEL)
return
}
w.Header().Set("Cache-Control", "max-age=2592000, public")
w.Write([]byte(info.ToJson()))
}
func getPublicFile(c *Context, w http.ResponseWriter, r *http.Request) {
c.RequireFileId()
if c.Err != nil {
return
}
if !c.App.Config().FileSettings.EnablePublicLink {
c.Err = model.NewAppError("getPublicFile", "api.file.get_public_link.disabled.app_error", nil, "", http.StatusNotImplemented)
return
}
info, err := c.App.GetFileInfo(c.Params.FileId)
if err != nil {
c.Err = err
return
}
hash := r.URL.Query().Get("h")
if len(hash) == 0 {
c.Err = model.NewAppError("getPublicFile", "api.file.get_file.public_invalid.app_error", nil, "", http.StatusBadRequest)
utils.RenderWebAppError(c.App.Config(), w, r, c.Err, c.App.AsymmetricSigningKey())
return
}
if subtle.ConstantTimeCompare([]byte(hash), []byte(app.GeneratePublicLinkHash(info.Id, *c.App.Config().FileSettings.PublicLinkSalt))) != 1 {
c.Err = model.NewAppError("getPublicFile", "api.file.get_file.public_invalid.app_error", nil, "", http.StatusBadRequest)
utils.RenderWebAppError(c.App.Config(), w, r, c.Err, c.App.AsymmetricSigningKey())
return
}
fileReader, err := c.App.FileReader(info.Path)
if err != nil {
c.Err = err
c.Err.StatusCode = http.StatusNotFound
}
defer fileReader.Close()
err = writeFileResponse(info.Name, info.MimeType, info.Size, fileReader, false, w, r)
if err != nil {
c.Err = err
return
}
}
func writeFileResponse(filename string, contentType string, contentSize int64, fileReader io.Reader, forceDownload bool, w http.ResponseWriter, r *http.Request) *model.AppError {
w.Header().Set("Cache-Control", "max-age=2592000, private")
w.Header().Set("X-Content-Type-Options", "nosniff")
if contentSize > 0 {
w.Header().Set("Content-Length", strconv.Itoa(int(contentSize)))
}
if contentType == "" {
contentType = "application/octet-stream"
} else {
for _, unsafeContentType := range UNSAFE_CONTENT_TYPES {
if strings.HasPrefix(contentType, unsafeContentType) {
contentType = "text/plain"
break
}
}
}
w.Header().Set("Content-Type", contentType)
var toDownload bool
if forceDownload {
toDownload = true
} else {
isMediaType := false
for _, mediaContentType := range MEDIA_CONTENT_TYPES {
if strings.HasPrefix(contentType, mediaContentType) {
isMediaType = true
break
}
}
toDownload = !isMediaType
}
filename = url.PathEscape(filename)
if toDownload {
w.Header().Set("Content-Disposition", "attachment;filename=\""+filename+"\"; filename*=UTF-8''"+filename)
} else {
w.Header().Set("Content-Disposition", "inline;filename=\""+filename+"\"; filename*=UTF-8''"+filename)
}
// prevent file links from being embedded in iframes
w.Header().Set("X-Frame-Options", "DENY")
w.Header().Set("Content-Security-Policy", "Frame-ancestors 'none'")
io.Copy(w, fileReader)
return nil
}