introduce new syntax aware spellchecking mechanism

Fallback to the old mechanism if no lexer is active.

Algorithm:
1. Wrap the active lexer's lex function and save it original value to restore it
2. Obtain the original token stream
3. Filter the tokens for visibility and if we want to spellcheck them (lexers.COMMENT and lexers.STRING by default)
4. Spellcheck the selected token. This is why this is unusably slow we shell out to our spellchecker for each token to check.
5. Split up the selected token into its correct and misspelled parts
6. Insert the correct parts unmodified and the misspelled ones as lexers.ERROR tokens into the token stream

Drawbacks:
1. It is unusably slow for now
2. The problem to decide which tokens should be spellchecked still remains.
   A lexer using different token names than we expect will prevent our spellchecking

TODO:
1. Introduce a way to disable syntax aware spellchecking and always use
   the old check full viewport approach
2. Speedup the syntax aware spellchecking (Maybe by splitting it into two iterations:
   First collect all toens to spellcheck and shell out once to spellcheck them all
   Second build up the token stream using the typo information we got)

0 files changed, 0 insertions, 0 deletions