
2018 @phenomnominal
I'm Craig.
I do JavaScript at

in New Zealand
2018 @phenomnominal
Let's start with a story...
2018 @phenomnominal

JSConf attendee!
Good Guy
You're a wizard!
2018 @phenomnominal
You went to Hogwarts!!

(School of Witchcraft and Wizardry)
Place where stuff happens
2018 @phenomnominal
You can do magic!!

This is a patronus
2018 @phenomnominal

You know Dumbledore!!!
Good Guy
2018 @phenomnominal
And Harry Potter is your Best Friend!!!!

Main Good Guy
2018 @phenomnominal
But there's a problem...

Maybe
Good?
Maybe Bad?
2018 @phenomnominal
Voldemort is back (again!)
Literally the
Worst Guy

2018 @phenomnominal
And he has put a curse on Harry!
Siriously
Evil.

2018 @phenomnominal
2018 @phenomnominal
If we don't save him in the next 25 minutes, Voldemort is going to kill him!
2018 @phenomnominal

This is bad.
Our friend is gone.
Time is running out.
2018 @phenomnominal
But there is one bit of good news...


2018 @phenomnominal
Harry speaks Parseltongue!

He can talk to snakes!
2018 @phenomnominal
Which, as every magical being knows, is a
Turing-complete programming language!
(this is 100% canon, don't look it up)
2018 @phenomnominal
Weakly Typed
Functional
Interpreted
Garbage collected
Significant whitespace
Parseltongue 🐍🐍🐍
2018 @phenomnominal
Parseltongue 🐍🐍🐍
Variable Declarations


Parseltongue:
JavaScript:
Only one type of variable
Always start with
Assignment Operator is
<~
sss
2018 @phenomnominal
Function Expressions


Parameters are defined with an array
Return uses the assignment operator
Significant whitespace
Parseltongue 🐍🐍🐍
2018 @phenomnominal
Parseltongue:
JavaScript:
Call Expression


Passing arguments to a function also uses the Assignment operator
Arguments are declared with an array
Parseltongue 🐍🐍🐍
2018 @phenomnominal
Parseltongue:
JavaScript:
Control Flow


Use an empty array when you have no arguments
Parseltongue 🐍🐍🐍
2018 @phenomnominal
Parseltongue:
JavaScript:
Loops

Range operator is
~>

Parseltongue 🐍🐍🐍
2018 @phenomnominal
Parseltongue:
JavaScript:
Spells
Parseltongue 🐍🐍🐍
2018 @phenomnominal
Parseltongue:
JavaScript:


is magic and becomes
sssCast
alert
2018 @phenomnominal
Harry's spell ⭐️⭐️⭐️
2018 @phenomnominal

Internet browsers don't understand Parseltongue.

2018 @phenomnominal
We need to translate Harry's Parseltongue into JavaScript!


2018 @phenomnominal



Parseltongue
JavaScript
???
2018 @phenomnominal



Parseltongue
JavaScript
REGEX!?
2018 @phenomnominal
Let's go a line at a time:

RegEx:
(the darkest of the dark arts)
2018 @phenomnominal
That gives us:

RegEx:
2018 @phenomnominal
Converting variables

2018 @phenomnominal
RegEx:
Converting loops

Hmmm...
2018 @phenomnominal
RegEx:
Converting functions

Ruh roh!
2018 @phenomnominal
RegEx:

2018 @phenomnominal
What are we really trying to do?
Transform one language into another

2018 @phenomnominal
Transpiling


(aka transfiguration)

2018 @phenomnominal
"Transpiling is a specific term for taking source code written in one language and transforming into another language that has a similar level of abstraction"
Transpiling
2018 @phenomnominal
Where do we start?
String literal
Function call
Identifier
Expression

2018 @phenomnominal
ESTree


2018 @phenomnominal
Esprima

2018 @phenomnominal
Abstract
disassociated from any specific instance
the way in which linguistic elements are put together
Syntax
TREE
data structure made up of vertices and edges without any cycles

2018 @phenomnominal

Wat.
2018 @phenomnominal

Wat.

2018 @phenomnominal
a data structure that represents the structure of code, without any actual syntax.
An AST is:
Code:


2018 @phenomnominal

a data structure that represents the structure of code, without any actual syntax.
An AST is:
AST:

2018 @phenomnominal
Which means...

2018 @phenomnominal
If we can get from Parseltongue to an AST...

Then we can go from that AST to JavaScript!
2018 @phenomnominal



Parseltongue
JavaScript
REGEX!?
2018 @phenomnominal



Parseltongue
JavaScript
AST
2018 @phenomnominal





Parseltongue
JavaScript
AST
???
???
2018 @phenomnominal
2018 @phenomnominal
Remember Esprima?

2018 @phenomnominal
Remember Esprima?
Esprima takes code written in a specific language (JavaScript) and turns it into an AST!
2018 @phenomnominal
We need to make our own Esprima for Parseltongue!


How do we do that?
2018 @phenomnominal







Parseltongue
JavaScript
AST
???
???
Tokens
Lexing
2018 @phenomnominal
Lexing/Tokenising
"Lexing is the process of breaking down source code into words that are relevant to the language, which are called tokens"
2018 @phenomnominal
Lexing/Tokenising
Tokens in JavaScript
Identifier
Keyword
Literal - bool, null, number, string
Punctuator
EOF
RegularExpression
Template

(we're getting a bit intense now - here's an owl)
2018 @phenomnominal
Lexing/Tokenising
Tokens in Parseltongue
Identifier
Keyword
Literal - bool, null, number, string
Punctuator
EOF
Indent
LineTerminator
Space

(we're getting a bit intense now - here's an owl)
2018 @phenomnominal
Identifier characters
Space
Punctuator characters
Space
Quote character
Lexing/Tokenising
2018 @phenomnominal

Punctuator
Punctuator
String characters
Quote character
2018 @phenomnominal

2018 @phenomnominal







Parseltongue
JavaScript
AST
Parsing
???
Tokens
Lexing
2018 @phenomnominal
Parsing

"Parsing is the process of taking the lexical tokens and applying the grammar of the language to them"
2018 @phenomnominal
Parsing
Variable Declaration
Identifier
Space
Punctuator -
Space
String literal


<~
2018 @phenomnominal
Identifier
Space
Punctuator -
Identifier
Punctuator -
Space
Identifier
Punctuator -
Parsing
Function Declaration

Line Terminator
Indent
[
,
]
2018 @phenomnominal
Parsing
Function Declaration

2018 @phenomnominal

Actual LOL
2018 @phenomnominal

2018 @phenomnominal
Now we just need to go from the AST to JavaScript...
2018 @phenomnominal







Parseltongue
JavaScript
AST
Parsing
Code Generating
Tokens
Lexing
2018 @phenomnominal
Code Generating

This is really hard to do by ourselves.
We can again stand on the shoulders of half-giants!
2018 @phenomnominal
ESCodeGen

2018 @phenomnominal

2018 @phenomnominal

2018 @phenomnominal
JavaScript!







Parseltongue
JavaScript
AST
Parsing
Code Generating
Tokens
Lexing
2018 @phenomnominal
We're almost ready to save Harry!
2018 @phenomnominal
But OH NO!
Voldemort can also speak Parseltongue!

2018 @phenomnominal
Voldemort will try to stop Harry from escaping!
He might even use the...
2018 @phenomnominal
Unforgivable Functions

2018 @phenomnominal
We need to come up with a way to stop Voldemort from using those functions!
2018 @phenomnominal








JavaScript
AST
Parsing
Code Generating
Tokens
Lexing
Parseltongue
???
2018 @phenomnominal








JavaScript
AST
Parsing
Code Generating
Tokens
Lexing
Parseltongue
Inspecting
2018 @phenomnominal
Inspecting
Transformation
Mutation
Validation








2018 @phenomnominal
ESQuery

2018 @phenomnominal
Linting
Detect bad stuff


2018 @phenomnominal
Linting
Fixing bad stuff

Tickling curse
2018 @phenomnominal
2018 @phenomnominal









JavaScript
AST
Parsing
Code Generating
Tokens
Lexing
Parseltongue
Inspection
2018 @phenomnominal
2018 @phenomnominal
We did it!
Harry is safe!

2018 @phenomnominal

All is well
(again)
2018 @phenomnominal
So...
2018 @phenomnominal

When do you actually use an AST?
2018 @phenomnominal
Ever had to change every file in your codebase?
2018 @phenomnominal
Ever had to generate boilerplate?
2018 @phenomnominal
Ever had to extract API documentation?
2018 @phenomnominal
You can use an AST for that!
(and probably should)
2018 @phenomnominal
ASTs help us with...
Performance
Maintenance
Consistency
Analysis
Robustness
Automation

2018 @phenomnominal
They're everywhere

2018 @phenomnominal
They're powerful

2018 @phenomnominal
They're useful

2018 @phenomnominal
Once you know where to find them
2018 @phenomnominal

Thanks!
Me: Craig
Twitter: @phenomnominal
Code: github.com/phenomnomnominal/parseltongue
Slides: slides.com/craigspence/fantastic-asts-and-where-to-find-them-jsconf
Resources:
https://github.com/dannysu/ecmascript1
https://github.com/estree/estree
https://github.com/jquery/esprima
https://github.com/estools/escodegen
http://estools.github.io/esquery
https://medium.com/@kosamari/how-to-be-a-compiler-make-a-compiler-with-javascript-4a8a13d473b4
https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/Parser_API
https://github.com/mozilla/source-map#generating-a-source-map
https://hacks.mozilla.org/2013/05/compiling-to-javascript-and-debugging-with-source-maps/
https://www.buzzfeed.com/zgalehouse/300-harry-potter-gifsthe-magic-never-ends-7sat
2018 @phenomnominal
Fantastic ASTs and Where To Find Them
By Craig Spence
Fantastic ASTs and Where To Find Them
JSConfAu
- 4,680