Upgrading server dependancies (#6431)

vendor/github.com/fsnotify/fsnotify/.travis.yml

@@ -2,12 +2,14 @@ sudo: false
 language: go
 
 go:
-  - 1.6.3
+  - 1.8
+  - 1.7.x
   - tip
 
 matrix:
   allow_failures:
     - go: tip
+  fast_finish: true
 
 before_script:
   - go get -u github.com/golang/lint/golint

vendor/github.com/fsnotify/fsnotify/CHANGELOG.md

@@ -79,7 +79,7 @@ kqueue: Fix logic for CREATE after REMOVE [#111](https://github.com/fsnotify/fsn
 
 ## v1.0.2 / 2014-08-17
 
-* [Fix] Missing create events on OS X. [#14](https://github.com/fsnotify/fsnotify/issues/14) (thanks @zhsso)
+* [Fix] Missing create events on macOS. [#14](https://github.com/fsnotify/fsnotify/issues/14) (thanks @zhsso)
 * [Fix] Make ./path and path equivalent. (thanks @zhsso)
 
 ## v1.0.0 / 2014-08-15
@@ -142,7 +142,7 @@ kqueue: Fix logic for CREATE after REMOVE [#111](https://github.com/fsnotify/fsn
 
 ## v0.9.2 / 2014-08-17
 
-* [Backport] Fix missing create events on OS X. [#14](https://github.com/fsnotify/fsnotify/issues/14) (thanks @zhsso)
+* [Backport] Fix missing create events on macOS. [#14](https://github.com/fsnotify/fsnotify/issues/14) (thanks @zhsso)
 
 ## v0.9.1 / 2014-06-12
 
@@ -161,7 +161,7 @@ kqueue: Fix logic for CREATE after REMOVE [#111](https://github.com/fsnotify/fsn
 ## v0.8.11 / 2013-11-02
 
 * [Doc] Add Changelog [#72][] (thanks @nathany)
-* [Doc] Spotlight and double modify events on OS X [#62][] (reported by @paulhammond)
+* [Doc] Spotlight and double modify events on macOS [#62][] (reported by @paulhammond)
 
 ## v0.8.10 / 2013-10-19
 

vendor/github.com/fsnotify/fsnotify/CONTRIBUTING.md

@@ -17,7 +17,7 @@ Please indicate that you have signed the CLA in your pull request.
 ### How fsnotify is Developed
 
 * Development is done on feature branches.
-* Tests are run on BSD, Linux, OS X and Windows.
+* Tests are run on BSD, Linux, macOS and Windows.
 * Pull requests are reviewed and [applied to master][am] using [hub][].
   * Maintainers may modify or squash commits rather than asking contributors to.
 * To issue a new release, the maintainers will:
@@ -44,7 +44,7 @@ This workflow is [thoroughly explained by Katrina Owen](https://splice.com/blog/
 
 ### Testing
 
-fsnotify uses build tags to compile different code on Linux, BSD, OS X, and Windows.
+fsnotify uses build tags to compile different code on Linux, BSD, macOS, and Windows.
 
 Before doing a pull request, please do your best to test your changes on multiple platforms, and list which platforms you were able/unable to test on.
 
@@ -58,7 +58,7 @@ To aid in cross-platform testing there is a Vagrantfile for Linux and BSD.
 
 Notice: fsnotify file system events won't trigger in shared folders. The tests get around this limitation by using the /tmp directory.
 
-Right now there is no equivalent solution for Windows and OS X, but there are Windows VMs [freely available from Microsoft](http://www.modern.ie/en-us/virtualization-tools#downloads).
+Right now there is no equivalent solution for Windows and macOS, but there are Windows VMs [freely available from Microsoft](http://www.modern.ie/en-us/virtualization-tools#downloads).
 
 ### Maintainers
 

vendor/github.com/fsnotify/fsnotify/README.md

@@ -8,14 +8,14 @@ fsnotify utilizes [golang.org/x/sys](https://godoc.org/golang.org/x/sys) rather
 go get -u golang.org/x/sys/...
 ```
 
-Cross platform: Windows, Linux, BSD and OS X.
+Cross platform: Windows, Linux, BSD and macOS.
 
 |Adapter   |OS        |Status    |
 |----------|----------|----------|
 |inotify   |Linux 2.6.27 or later, Android\*|Supported [![Build Status](https://travis-ci.org/fsnotify/fsnotify.svg?branch=master)](https://travis-ci.org/fsnotify/fsnotify)|
-|kqueue    |BSD, OS X, iOS\*|Supported [![Build Status](https://travis-ci.org/fsnotify/fsnotify.svg?branch=master)](https://travis-ci.org/fsnotify/fsnotify)|
+|kqueue    |BSD, macOS, iOS\*|Supported [![Build Status](https://travis-ci.org/fsnotify/fsnotify.svg?branch=master)](https://travis-ci.org/fsnotify/fsnotify)|
 |ReadDirectoryChangesW|Windows|Supported [![Build status](https://ci.appveyor.com/api/projects/status/ivwjubaih4r0udeh/branch/master?svg=true)](https://ci.appveyor.com/project/NathanYoungman/fsnotify/branch/master)|
-|FSEvents  |OS X          |[Planned](https://github.com/fsnotify/fsnotify/issues/11)|
+|FSEvents  |macOS         |[Planned](https://github.com/fsnotify/fsnotify/issues/11)|
 |FEN       |Solaris 11    |[In Progress](https://github.com/fsnotify/fsnotify/issues/12)|
 |fanotify  |Linux 2.6.37+ | |
 |USN Journals |Windows    |[Maybe](https://github.com/fsnotify/fsnotify/issues/53)|
@@ -23,7 +23,7 @@ Cross platform: Windows, Linux, BSD and OS X.
 
 \* Android and iOS are untested.
 
-Please see [the documentation](https://godoc.org/github.com/fsnotify/fsnotify) for usage. Consult the [Wiki](https://github.com/fsnotify/fsnotify/wiki) for the FAQ and further information.
+Please see [the documentation](https://godoc.org/github.com/fsnotify/fsnotify) and consult the [FAQ](#faq) for usage information.
 
 ## API stability
 
@@ -41,6 +41,35 @@ Please refer to [CONTRIBUTING][] before opening an issue or pull request.
 
 See [example_test.go](https://github.com/fsnotify/fsnotify/blob/master/example_test.go).
 
+## FAQ
+
+**When a file is moved to another directory is it still being watched?**
+
+No (it shouldn't be, unless you are watching where it was moved to).
+
+**When I watch a directory, are all subdirectories watched as well?**
+
+No, you must add watches for any directory you want to watch (a recursive watcher is on the roadmap [#18][]).
+
+**Do I have to watch the Error and Event channels in a separate goroutine?**
+
+As of now, yes. Looking into making this single-thread friendly (see [howeyc #7][#7])
+
+**Why am I receiving multiple events for the same file on OS X?**
+
+Spotlight indexing on OS X can result in multiple events (see [howeyc #62][#62]). A temporary workaround is to add your folder(s) to the *Spotlight Privacy settings* until we have a native FSEvents implementation (see [#11][]).
+
+**How many files can be watched at once?**
+
+There are OS-specific limits as to how many watches can be created:
+* Linux: /proc/sys/fs/inotify/max_user_watches contains the limit, reaching this limit results in a "no space left on device" error.
+* BSD / OSX: sysctl variables "kern.maxfiles" and "kern.maxfilesperproc", reaching these limits results in a "too many open files" error.
+
+[#62]: https://github.com/howeyc/fsnotify/issues/62
+[#18]: https://github.com/fsnotify/fsnotify/issues/18
+[#11]: https://github.com/fsnotify/fsnotify/issues/11
+[#7]: https://github.com/howeyc/fsnotify/issues/7
+
 [contributing]: https://github.com/fsnotify/fsnotify/blob/master/CONTRIBUTING.md
 
 ## Related Projects

vendor/github.com/fsnotify/fsnotify/fsnotify.go

@@ -9,6 +9,7 @@ package fsnotify
 
 import (
 	"bytes"
+	"errors"
 	"fmt"
 )
 
@@ -60,3 +61,6 @@ func (op Op) String() string {
 func (e Event) String() string {
 	return fmt.Sprintf("%q: %s", e.Name, e.Op.String())
 }
+
+// Common errors that can be reported by a watcher
+var ErrEventOverflow = errors.New("fsnotify queue overflow")

vendor/github.com/fsnotify/fsnotify/inotify.go

@@ -24,7 +24,6 @@ type Watcher struct {
 	Events   chan Event
 	Errors   chan error
 	mu       sync.Mutex // Map access
-	cv       *sync.Cond // sync removing on rm_watch with IN_IGNORE
 	fd       int
 	poller   *fdPoller
 	watches  map[string]*watch // Map of inotify watches (key: path)
@@ -56,7 +55,6 @@ func NewWatcher() (*Watcher, error) {
 		done:     make(chan struct{}),
 		doneResp: make(chan struct{}),
 	}
-	w.cv = sync.NewCond(&w.mu)
 
 	go w.readEvents()
 	return w, nil
@@ -103,21 +101,23 @@ func (w *Watcher) Add(name string) error {
 	var flags uint32 = agnosticEvents
 
 	w.mu.Lock()
-	watchEntry, found := w.watches[name]
-	w.mu.Unlock()
-	if found {
-		watchEntry.flags |= flags
-		flags |= unix.IN_MASK_ADD
+	defer w.mu.Unlock()
+	watchEntry := w.watches[name]
+	if watchEntry != nil {
+		flags |= watchEntry.flags | unix.IN_MASK_ADD
 	}
 	wd, errno := unix.InotifyAddWatch(w.fd, name, flags)
 	if wd == -1 {
 		return errno
 	}
 
-	w.mu.Lock()
-	w.watches[name] = &watch{wd: uint32(wd), flags: flags}
-	w.paths[wd] = name
-	w.mu.Unlock()
+	if watchEntry == nil {
+		w.watches[name] = &watch{wd: uint32(wd), flags: flags}
+		w.paths[wd] = name
+	} else {
+		watchEntry.wd = uint32(wd)
+		watchEntry.flags = flags
+	}
 
 	return nil
 }
@@ -135,6 +135,13 @@ func (w *Watcher) Remove(name string) error {
 	if !ok {
 		return fmt.Errorf("can't remove non-existent inotify watch for: %s", name)
 	}
+
+	// We successfully removed the watch if InotifyRmWatch doesn't return an
+	// error, we need to clean up our internal state to ensure it matches
+	// inotify's kernel state.
+	delete(w.paths, int(watch.wd))
+	delete(w.watches, name)
+
 	// inotify_rm_watch will return EINVAL if the file has been deleted;
 	// the inotify will already have been removed.
 	// watches and pathes are deleted in ignoreLinux() implicitly and asynchronously
@@ -152,13 +159,6 @@ func (w *Watcher) Remove(name string) error {
 		return errno
 	}
 
-	// wait until ignoreLinux() deleting maps
-	exists := true
-	for exists {
-		w.cv.Wait()
-		_, exists = w.watches[name]
-	}
-
 	return nil
 }
 
@@ -245,13 +245,31 @@ func (w *Watcher) readEvents() {
 
 			mask := uint32(raw.Mask)
 			nameLen := uint32(raw.Len)
+
+			if mask&unix.IN_Q_OVERFLOW != 0 {
+				select {
+				case w.Errors <- ErrEventOverflow:
+				case <-w.done:
+					return
+				}
+			}
+
 			// If the event happened to the watched directory or the watched file, the kernel
 			// doesn't append the filename to the event, but we would like to always fill the
 			// the "Name" field with a valid filename. We retrieve the path of the watch from
 			// the "paths" map.
 			w.mu.Lock()
-			name := w.paths[int(raw.Wd)]
+			name, ok := w.paths[int(raw.Wd)]
+			// IN_DELETE_SELF occurs when the file/directory being watched is removed.
+			// This is a sign to clean up the maps, otherwise we are no longer in sync
+			// with the inotify kernel state which has already deleted the watch
+			// automatically.
+			if ok && mask&unix.IN_DELETE_SELF == unix.IN_DELETE_SELF {
+				delete(w.paths, int(raw.Wd))
+				delete(w.watches, name)
+			}
 			w.mu.Unlock()
+
 			if nameLen > 0 {
 				// Point "bytes" at the first byte of the filename
 				bytes := (*[unix.PathMax]byte)(unsafe.Pointer(&buf[offset+unix.SizeofInotifyEvent]))
@@ -262,7 +280,7 @@ func (w *Watcher) readEvents() {
 			event := newEvent(name, mask)
 
 			// Send the events that are not ignored on the events channel
-			if !event.ignoreLinux(w, raw.Wd, mask) {
+			if !event.ignoreLinux(mask) {
 				select {
 				case w.Events <- event:
 				case <-w.done:
@@ -279,15 +297,9 @@ func (w *Watcher) readEvents() {
 // Certain types of events can be "ignored" and not sent over the Events
 // channel. Such as events marked ignore by the kernel, or MODIFY events
 // against files that do not exist.
-func (e *Event) ignoreLinux(w *Watcher, wd int32, mask uint32) bool {
+func (e *Event) ignoreLinux(mask uint32) bool {
 	// Ignore anything the inotify API says to ignore
 	if mask&unix.IN_IGNORED == unix.IN_IGNORED {
-		w.mu.Lock()
-		defer w.mu.Unlock()
-		name := w.paths[int(wd)]
-		delete(w.paths, int(wd))
-		delete(w.watches, name)
-		w.cv.Broadcast()
 		return true
 	}
 

vendor/github.com/fsnotify/fsnotify/inotify_test.go

@@ -293,25 +293,23 @@ func TestInotifyRemoveTwice(t *testing.T) {
 		t.Fatalf("Failed to add testFile: %v", err)
 	}
 
-	err = os.Remove(testFile)
+	err = w.Remove(testFile)
 	if err != nil {
-		t.Fatalf("Failed to remove testFile: %v", err)
+		t.Fatalf("wanted successful remove but got:", err)
 	}
 
 	err = w.Remove(testFile)
 	if err == nil {
 		t.Fatalf("no error on removing invalid file")
 	}
-	s1 := fmt.Sprintf("%s", err)
 
-	err = w.Remove(testFile)
-	if err == nil {
-		t.Fatalf("no error on removing invalid file")
+	w.mu.Lock()
+	defer w.mu.Unlock()
+	if len(w.watches) != 0 {
+		t.Fatalf("Expected watches len is 0, but got: %d, %v", len(w.watches), w.watches)
 	}
-	s2 := fmt.Sprintf("%s", err)
-
-	if s1 != s2 {
-		t.Fatalf("receive different error - %s / %s", s1, s2)
+	if len(w.paths) != 0 {
+		t.Fatalf("Expected paths len is 0, but got: %d, %v", len(w.paths), w.paths)
 	}
 }
 
@@ -358,3 +356,94 @@ func TestInotifyInnerMapLength(t *testing.T) {
 		t.Fatalf("Expected paths len is 0, but got: %d, %v", len(w.paths), w.paths)
 	}
 }
+
+func TestInotifyOverflow(t *testing.T) {
+	// We need to generate many more events than the
+	// fs.inotify.max_queued_events sysctl setting.
+	// We use multiple goroutines (one per directory)
+	// to speed up file creation.
+	numDirs := 128
+	numFiles := 1024
+
+	testDir := tempMkdir(t)
+	defer os.RemoveAll(testDir)
+
+	w, err := NewWatcher()
+	if err != nil {
+		t.Fatalf("Failed to create watcher: %v", err)
+	}
+	defer w.Close()
+
+	for dn := 0; dn < numDirs; dn++ {
+		testSubdir := fmt.Sprintf("%s/%d", testDir, dn)
+
+		err := os.Mkdir(testSubdir, 0777)
+		if err != nil {
+			t.Fatalf("Cannot create subdir: %v", err)
+		}
+
+		err = w.Add(testSubdir)
+		if err != nil {
+			t.Fatalf("Failed to add subdir: %v", err)
+		}
+	}
+
+	errChan := make(chan error, numDirs*numFiles)
+
+	for dn := 0; dn < numDirs; dn++ {
+		testSubdir := fmt.Sprintf("%s/%d", testDir, dn)
+
+		go func() {
+			for fn := 0; fn < numFiles; fn++ {
+				testFile := fmt.Sprintf("%s/%d", testSubdir, fn)
+
+				handle, err := os.Create(testFile)
+				if err != nil {
+					errChan <- fmt.Errorf("Create failed: %v", err)
+					continue
+				}
+
+				err = handle.Close()
+				if err != nil {
+					errChan <- fmt.Errorf("Close failed: %v", err)
+					continue
+				}
+			}
+		}()
+	}
+
+	creates := 0
+	overflows := 0
+
+	after := time.After(10 * time.Second)
+	for overflows == 0 && creates < numDirs*numFiles {
+		select {
+		case <-after:
+			t.Fatalf("Not done")
+		case err := <-errChan:
+			t.Fatalf("Got an error from file creator goroutine: %v", err)
+		case err := <-w.Errors:
+			if err == ErrEventOverflow {
+				overflows++
+			} else {
+				t.Fatalf("Got an error from watcher: %v", err)
+			}
+		case evt := <-w.Events:
+			if !strings.HasPrefix(evt.Name, testDir) {
+				t.Fatalf("Got an event for an unknown file: %s", evt.Name)
+			}
+			if evt.Op == Create {
+				creates++
+			}
+		}
+	}
+
+	if creates == numDirs*numFiles {
+		t.Fatalf("Could not trigger overflow")
+	}
+
+	if overflows == 0 {
+		t.Fatalf("No overflow and not enough creates (expected %d, got %d)",
+			numDirs*numFiles, creates)
+	}
+}

vendor/github.com/fsnotify/fsnotify/integration_darwin_test.go

@@ -13,9 +13,9 @@ import (
 	"golang.org/x/sys/unix"
 )
 
-// testExchangedataForWatcher tests the watcher with the exchangedata operation on OS X.
+// testExchangedataForWatcher tests the watcher with the exchangedata operation on macOS.
 //
-// This is widely used for atomic saves on OS X, e.g. TextMate and in Apple's NSDocument.
+// This is widely used for atomic saves on macOS, e.g. TextMate and in Apple's NSDocument.
 //
 // See https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man2/exchangedata.2.html
 // Also see: https://github.com/textmate/textmate/blob/cd016be29489eba5f3c09b7b70b06da134dda550/Frameworks/io/src/swap_file_data.cc#L20