Fixes #1079
Fixes #1080
Fixes #1084
Fixes #1091
Fixes #1107

This PR does one main thing: it adds support for handling file updates without watchers.
It took the better part of an 8 hour session, but I figured it out.
I didn't even have to drop out of watch programs to do it.

There is a bit of noise on this PR, and I'm sorry for that. The important changes are on fa963c8.
f28e4dd was a quick refactor I did to help me look at the problem properly - all I did was move the various program creation functions we have into their own folders.

The solution in a nutshell.

1. The realisation that I can override the setTimeout function that the watch program uses. This let me turn the async 250ms queued updates into synchronous updates, which solved the huge problem with how it handled directory updates.

   • typescript-eslint/packages/typescript-estree/src/create-program/createWatchProgram.ts

     Lines 233 to 248 in 78025a3

     	/*
     	* The watch change callbacks TS provides us all have a 250ms delay before firing
     	* https://github.com/microsoft/TypeScript/blob/b845800bdfcc81c8c72e2ac6fdc2c1df0cdab6f9/src/compiler/watch.ts#L1013
     	*
     	* We live in a synchronous world, so we can't wait for that.
     	* This is a bit of a hack, but it lets us immediately force updates when we detect a tsconfig or directory change
     	*/
     	const oldSetTimeout = watchCompilerHost.setTimeout;
     	watchCompilerHost.setTimeout = (cb, ms, ...args): unknown => {
     	if (ms === 250) {
     	cb();
     	return null;
     	}
     	return oldSetTimeout && oldSetTimeout(cb, ms, ...args);
     	};

2. Using fs.stat to check for tsconfig changes instead of watching for changes.

   • typescript-eslint/packages/typescript-estree/src/create-program/createWatchProgram.ts

     Lines 253 to 266 in 78025a3

     	const tsconfigStatTimestamp = new Map<string, number>();
     	function hasTSConfigChanged(tsconfigPath: string): boolean {
     	const stat = fs.statSync(tsconfigPath);
     	const lastModifiedAt = stat.mtimeMs;
     	const cachedLastModifiedAt = tsconfigStatTimestamp.get(tsconfigPath);
     	tsconfigStatTimestamp.set(tsconfigPath, lastModifiedAt);
     	if (cachedLastModifiedAt === undefined) {
     	return false;
     	}
     	return Math.abs(cachedLastModifiedAt - lastModifiedAt) > Number.EPSILON;
     	}

   • typescript-eslint/packages/typescript-estree/src/create-program/createWatchProgram.ts

     Lines 273 to 282 in 78025a3

     	if (hasTSConfigChanged(tsconfigPath)) {
     	/*
     	* If the stat of the tsconfig has changed, that could mean the include/exclude/files lists has changed
     	* We need to make sure typescript knows this so it can update appropriately
     	*/
     	log('tsconfig has changed - triggering program update. %s', tsconfigPath);
     	fileWatchCallbackTrackingMap
     	.get(tsconfigPath)!
     	.forEach(cb => cb(tsconfigPath, ts.FileWatcherEventKind.Changed));
     	}

3. Manually triggering a directory update if the file isn't in the folder

   • typescript-eslint/packages/typescript-estree/src/create-program/createWatchProgram.ts

     Lines 294 to 324 in 78025a3

     	/*
     	* Missing source file means our program's folder structure might be out of date.
     	* So we need to tell typescript it needs to update the correct folder.
     	*/
     	log('File was not found in program - triggering folder update. %s', filePath);
     	// Find the correct directory callback by climbing the folder tree
     	let current: string | null = null;
     	let next: string | null = path.dirname(filePath);
     	let hasCallback = false;
     	while (current !== next) {
     	current = next;
     	const folderWatchCallbacks = folderWatchCallbackTrackingMap.get(current);
     	if (folderWatchCallbacks) {
     	folderWatchCallbacks.forEach(cb =>
     	cb(current!, ts.FileWatcherEventKind.Changed),
     	);
     	hasCallback = true;
     	break;
     	}
     	next = path.dirname(current);
     	}
     	if (!hasCallback) {
     	/*
     	* No callback means the paths don't matchup - so no point returning any program
     	* this will signal to the caller to skip this program
     	*/
     	log('No callback found for file, not part of this program. %s', filePath);
     	return null;
     	}

4. Manually triggering file deletions to support file renaming. This one feels like a bug in the watch program. It assumes if the number of files in the program doesn't change, then the program hasn't changed. If you rename a file, the number of files in the program doesn't change, but the list ofc doesn't match the new list of files reported. I think the idea is that it's expecting a file watcher to signal a deletion when a file is renamed. Regardless, this is a last resort to manually trigger file deletions when it thinks the file is renamed.

   • typescript-eslint/packages/typescript-estree/src/create-program/createWatchProgram.ts

     Lines 333 to 363 in 78025a3

     	/*
     	* At this point we're in one of two states:
     	* - The file isn't supposed to be in this program due to exclusions
     	* - The file is new, and was renamed from an old, included filename
     	*
     	* For the latter case, we need to tell typescript that the old filename is now deleted
     	*/
     	log(
     	'File was still not found in program after directory update - checking file deletions. %s',
     	filePath,
     	);
     	const rootFilenames = updatedProgram.getRootFileNames();
     	// use find because we only need to "delete" one file to cause typescript to do a full resync
     	const deletedFile = rootFilenames.find(file => !fs.existsSync(file));
     	if (!deletedFile) {
     	// There are no deleted files, so it must be the former case of the file not belonging to this program
     	return null;
     	}
     	const fileWatchCallbacks = fileWatchCallbackTrackingMap.get(deletedFile);
     	if (!fileWatchCallbacks) {
     	// shouldn't happen, but just in case
     	log('Could not find watch callbacks for root file. %s', deletedFile);
     	return updatedProgram;
     	}
     	log('Marking file as deleted. %s', deletedFile);
     	fileWatchCallbacks.forEach(cb =>
     	cb(deletedFile, ts.FileWatcherEventKind.Deleted),
     	);

If my assumptions are correct, the cost of all of this should only be paid in watch mode, when a new file is encountered.

Other notes:

• Refactored to isolate all the program creation functions
• Added more logging calls (it was easier to use logging than always attempting to attach a debugger). I left them in because I figured it might help us in future.
• Added (slightly) more complex unit tests to ensure both deeply nested file creation, and cross folder file renaming actually works.
• Fixed a bug in the rule tester with it not respecting locally defined test case filenames (it only used the filename from the top level, which broke tests).
• Stopped passing both options and extra into the functions by adding filePath to extra.

Video of it working perfectly in vscode: https://youtu.be/sg54KuUdw9U