Translating Transpilating
Learning your first programming language can lead to tunnel vision. Sometimes the intricacies and rules of the language can obfuscate the necessity to understand the language’s purpose and how it gives instructions to machines. As an engineer it is important to be language agnostic and see the language as a set of tools rather than a religion. JavaScript is my first language and currently I still see it more as a rule set rather than a tool. The language’s mechanical interplay with the browser and my local machine is mysterious and the advantages/disadvantages it has with other languages, even more so.
The introduction of Babel, React, and AngularJS to my dev environment has sparked my interest in how JavaScript transforms to suit needs and expand features. Investigating further led to a broader education in programming languages and a deeper context for how JavaScript fits within that conversation. In order to take advantage of these new frameworks languages are transpiled to JavaScript so they can work within the browser.
Transpiling versus Compiling
Inevitably a discussion of transpiling must address the concerns of compiling programming languages and what differences, if any, are there between transpiling and compiling. Unfortunately, an in-depth look into compiling can spiral into topics like ‘compilation versus interpretation’ and ‘just-in-time compilation’ and further topics that lie out of the direct purview of transpiling. Even more confusingly, the two terms are used interchangeably frequently obscuring their differences. In order to proceed with an investigation of transpilation without getting too caught up in the weeds of compilation we will distinguish the two.
While the definitions can be imprecise, the distinctions between transpiling and compiling generally revolve around what the level of computer language that the input language is being transformed into. If the language is a High Level Programming language e.g. JavaScript, meaning that the language is abstracted enough that it is not concerned with the machine hardware, and it is being transformed to a Low-Level Programming Language, e.g. Machine Code, meaning that the programming language is concerned with giving instructions that a machine can read, it is considered compiling.
An example of compiling is how JavaScript is actually interpreted by the browser. Chrome utilizes their V8 WebAssembly engine to compile and render JavaScript. The engine is built in C++ and works with multiple parsers and two separate compilers to achieve translating JavaScript so it can be ran in the browser. A diagram that illustrate this compilation follows below, this information derives from Franziska Hinkelmann, a Senior Engineer on Google’s team who gave a great talk at JSConf about Chrome’s V8 engine which can be viewed here.
A simplistic way to put it is that compiling takes programming language and makes it more legible for the machine. I took my own code and utilized node to compile it from JavaScript to bytecode with this command
~$ node --print-bytecode byte.jsWhile the code was simply a console.log statement with a few dependencies, I was bombarded with the following output.
A not so brief way to understand a simple code snippet. Compiling facilitates this interpretation.
Transpiling, on the other hand, is deemed a source-code to source-code translator, meaning it is a way to translate a language to a language of comparable level of abstraction from the machine language. Most often, transpiling is used to describe transforming one high-level human readable programming language to another.
Why Transpile code?
The reasons for transpiling/ intergrating a transpiler are numerous. Transpilers allow access to features of one language not available in the source language natively. JavaScript as a nascent language, before the advent of es6, had many shortcomings that inconvenienced developers moving to web development from other languages. One such shortcoming was the fact that it JavaScript a loosely typed language. This means that when declaring variables, the variable’s type is not also declared. This flexibility can lead to bugs and TypeScript was built with type annotation and compile-time type checking to enable developers to have more control over type within their code. Additionally, TypeScript is a superset of JavaScript meaning that it has all the features of JavaScript and is updated to meet the ECMAScript standard. TypeScript is a language that must transpile to JavaScript before it can be run, languages like this are called “compile-to-JS” languages.
Another “compile-to-JS” language, CoffeeScript, illustrates a more ignominious need to transpile. As chronicled in the story, “The Great CoffeeScript to Typescript Migration of 2017, ” written by DropBox engineer David Goldstein, DropBox was built on Python and CoffeeScript was a more analogous to Python than the pre es6 vanilla JavaScript. Unfortunately, features of CoffeeScript particularly relating to whitespace being integral to the programming led to hard to find bugs and the need to transpile DropBox’s full codebase into TypeScript. Sometimes transpilation is necessary to update outmoded or buggy codebases.
Besides integrating new features into JavaScript, another important use for transpilers is accessibility. Developers want their apps and sites to have a uniform look and feel to every customer who may encounter them. Unfortunately, the web ecosystem is plentiful with web browsers and versions of those browsers that may not be fully functional with the newest version of JavaScript. It is imperative that developers are forward-thinking and use transpilation to make their sites backwards compatible with browsers that may not take advantage of the features that make up the infrastructure of the app. This does not end with es6 as JavaScript is always evolving developers may want to use feature of the so-called esNext (the features to come from the ever forthcoming ECMAScript) and customers need to be able to experience any advances afforded by new features. For context to browsers and compatibility, check out “The ECMAScript Compatibility Table”
Furthermore, frameworks such as JSX in React and Angular use transpilation to JavaScript to power their tools in a way that supercharges the front-end capabilities of JavaScript. For more on the potential benefits of transpiling and integrating ‘compile-to-JS’ languages see “What JavaScript Programmers Need To Know About Transpilation”
How Transpilation Works
The mechanics of transpilers and compilers can be complex. A general overview of the procedure is that a parser is used to decouple the source language’s code from its syntax while keeping reference to the meaningful bits of the code, its lexemes. Through this parsing process the code is converted into a tree structure known as an Abstract Syntax Tree (AST). From here, the AST is traversed and the code is reconfigured to fit into the syntax of the language that the source is being transpiled to and then the code proceeds to be generated and optimized. Obviously, different languages have separate syntax so the mechanism and functions that facilitate the parsing differ between different sources and destinations.
The mechanics of transpilation are quite complex and deserve further investigation. Incredible resources into the programming behind transpilation are the GitHub repository ‘the super tiny compiler’ and ‘Transpiling between any programming languages’ from the MongoDB engineering journal. A couple interactive resources are the Babel Repl which is a repl that showcases the capabilities of the popular JavaScript transpiler Babel and Esprima that enables building syntax trees that allow for analysis of programming syntax.
Web and app development by necessity are inextricable from change as customers and developers must evolve as quickly as habits and platforms change. JavaScript as the common denominator of web development, must be flexible to the needs of developers. Sometimes the built-in conventions of JavaScript or the accessibility of its newest features inhibit this need for flexibility. This is where the ability to transpile JavaScript comes into play extending its functionality in a variety of ways.
resources and further reading:
https://thenewstack.io/javascript-transpilers-need-know/
https://kangax.github.io/compat-table/es6/