If you could develop your own programming language from scratch, what would it look like? If you’re a C programmer, maybe you’d want something that was a little less painful to use. If you’re a Python programmer, maybe you’d create something that looked like Python but ran faster.

This was the question that Julia’s creators asked themselves when they first conceived the language in 2009. They wanted an open source language that was fast, but easy to program, and could handle a range of tasks from numeric computing to string processing without having to make any compromises.

They launched the first version in 2012, and since then it has become the little language that could. With v1.2 nearly done it’s time for a closer look.

The best of all worlds

In the past, people wanting to do some heavy mathematical processing might prototype their code in something like Python but then transition to a lower-level-language like C for the high performance parts. In Julia, the idea is to do everything in the same language. This even forms part of its tagline: “Looks like Python, feels like Lisp, runs like Fortran”.

Julia has a lot in common with Python. It’s scripted and interactive, with dynamic typing for flexibility, but it is JIT compiled. However, it has some differences.

One of these is multiple dispatch, which speeds up performance by enabling coders to add methods to generic functions. A function can do specific things with different data types if we specify the data type using a double colon. So we could define multiple methods for the same function foo depending on type, and all would coexist. Which one  runs depends on which argument type you pass in.

foo(bar::Int) = “It’s a number!”

foo(bar::String) = “It’s a string!”

foo(bar::Array) = “It’s an array!”

foo(1)

“It’s a number!”

foo(“One”)

“It’s a string!”

foo([1,1,1,1])

“It’s an array!”

It’s features like this that enable Julia’s performance benchmarks to show a close correlation with C for such a high-level, scripting-type language. Another advantage is that it is designed to be highly distributed across large numbers of nodes. It includes built-in parallel computing support including (currently experimental) multi-threading functionality and an implementation of multi-core distributed memory processing.

This has got it into some pretty heavyweight projects. For example,  last year it was the language of choice for Celeste, a project to catalogue 1889 million astronomical objects from the Sloan Digital Sky Survey, in a dataset measuring 178 TB. It ran in 1.3m threads on 9,300 nodes, taking just under 15 minutes to process the lot. That’s the kind of statistic that should make people sit up and take notice.

In spite of this, it’s still a well-kept secret compared to its older brethren, according to the indexes. Tiobe’s November 2018 rankings put it at 40th, trailing Ada, Prolog and Fortran. PYPL, which analyses how often language tutorials are searched on Google, has it in 22nd place, moving up 0.26 per cent from 2017 and beating out only Delphi in the rankings. Analyst RedMonk’s most recent analysis shows it ranking relatively highly in the number of related GitHub projects, but scoring poorly in Stack Overflow discussions. In short, people seem to be doing it, rather than talking about it, at least on that site.

Those folks doing stuff with Julia include not just tech giants like Amazon, Facebook, Apple, Microsoft and Google, but other companies across different industries including Ford, Comcast, Disney, Capital One and NASA.

Using it

So how do you use Python with Julia? Juno is a Julia IDE based on the Atom text editor, and comes as part of the JuliaPro Julia distribution. It has inline results, a REPL, and a plot plane, which statisticians and data scientists will love. Alternatively, there is an extension for the free Visual Studio Code IDE, alongside a plugin for SublimeText. Diehards will also appreciate the plugins for Vim and Emacs.

Alternatively, there’s a Jupyter kernel for Julia, which means that you can run Jupyter notebooks for the language on your machine. You can use these directly in the browser via JuliaBox, which is an online service hosted by Julia Computing, the company that co-ordinates and promotes Julia. You can log into this using email, Google, LinkedIn or GitHub. It gives you a free online sandbox to experiment with, and the ability to pay for more compute if you want to do some heavy lifting.

Syntax

When it comes to Syntax, Julia really does look a lot like Python, but with some small differences. For example, it uses 1-based indexing. Array numbering begins at [1] in Python, whereas Python’s equivalent lists begin at [0].

Also, unlike Python, which is obsessed with proper indentation, Julia doesn’t care and won’t enforce it (but won’t stop you using it, either). I prefer Python’s approach here, as it keeps program structure easier to read.

There are other small differences, like the position of step counters in range operations.  Counting from four to 10 in increments of two looks like this in Python:

for i in range(4, 10, 2):

And in Julia:

for i in 4:2:10

Julia doesn’t use colons for its loops. It’s all slightly more elegant, as are ternary conditionals. In Python:

age = 15

print(‘kid’ if age < 18 else ‘adult’)

Python 3 (which we’re using here) forces the use of quotes in a print statement, whereas Python 2 doesn’t.

In Julia:

age = 15

age < 18 ? “kid” : “adult”

When it comes to functions, you use function in Julia rather than Python’s def, although you can dispense with it altogether for one-line functions:

squareit(x) = x*x

And there is no need for return statements in Julia, because it returns whatever is on the last line, whereas Python returns the None type.

Julia also has lambda functions as Python does, which you’d use to pass as arguments to higher-order functions. It uses a different syntax, though:

In Julia:

a -> a*a

In Python:

lambda a : a*a

Packages

Julia has its own package manager, which is curated and managed by Julia Computing, and it has its pros and cons when compared to Python. In Python, you’ll install packages with something like Pip, but it’s best practice to create different virtual environments for each project using the virtualenv environment manager or something equivalent. This lets you manage different projects with different versions of packages and dependencies. It also lets you manage different versions of Python. The Python community still straddles Python 2 and 3, with many using both.

In Julia, the Pkg package manager automatically collects different sets of packages into environments, managed by manifest files. These environments can also overlay each other, meaning that you can combine specific environments with different toolsets.

Adding a package (eg the Example package) is easy, although you must authenticate by logging into the Julia registry using your email, Google or GitHub credentials the first time (Juno will display a HTML login page).

Then, add a package directly from the REPL by hitting ] to drop into package mode. Afterwards  just type

Add Example

Alternatively, in non-package REPL mode or in a Julia program you have to call the package manager first:

Using Pkg

Pkg.add(“Example”)

Uses Python packages

Julia has over 1900 packages of its own, and you can browse these using Julia Observer.

There’s a lot of functionality in there, but you might not find everything you need, or you might just really like that old Python package that you used to use.

Julia can add packages from other languages. It supports Python, R and C libraries. You get at Python modules by installing a Julia package called PyCall that uses the installed Python library.

using Pkg

Pkg.add(“PyCall”)

using PyCall

You can also change the Python version you’re using by altering the environment path. Then use @pyimport to get the Python package you want. The PyCall package documentation shows how you can use it to import the standard math package and then use one of its functions.

@pyimport math

math.sin(math.pi / 4) – sin(pi / 4)

So what’s that @ symbol for? That highlights another aspect of Python known as metaprogramming. Remember the part of the tagline that says Julia feels like Lisp? Metaprogramming is one reason why.

At some point, most coders have run across a situation where they wished they could use their code to write other code. Producing a bunch of functions that do very similar things is a good example. If you find yourself dreaming about how to concatenate variables into strings to build commands, then this feature will appeal to you.

Julia is homoiconic, meaning that it can manipulate code as a data structure. A basic piece of code can be referenced as an expression by putting it in parentheses, prefixed by a colon. So:

anexpression = :(a + b)

You can see how Julia breaks that expression down into its constituent parts by dumping it:

dump(anexpression)

Expr

 head: Symbol call

 args: Array{Any}((3,))

   1: Symbol +

   2: Symbol a

   3: Symbol b

These expressions can be passed to macro commands, which are prefixed with macro rather than function.

macro mycheck(myinput)

   println(“$myinput is an expression and here are its arguments”)

   println(myinput.args)

end

@mycheck a + b

a + b is an expression and here are its arguments

Any[:+, :a, :b]

See how the @mycheck macro evaluated myinput as an expression rather than as a pair of literal variables?

Macro expressions evaluate expressions when the compiler parses the code, rather than when it actually runs it. This enables them to unpack those macro expressions and insert them fully into the code. It’s like giving an envelope with ‘code in here’ on the front, and a code snippet inside. The macro puts it on a shelf, and then when the compiler calls upon it to run, it opens the envelope and inserts that code where it’s meant to go before it executes.

Metaprogramming and macros enable you to write macros that change based on the code that you pass to them. You could create a skeleton macro that offered a different kind of execution loop, for example. The Introducing Julia Wikibook gives an example of using an @until macro to create an Until loop, which doesn’t exist natively in Julia. The macro takes a conditional and a code block passed by the user.  Or you could pass an array of expressions as code snippets to a macro, enabling it to do slightly different things each time.

Python isn’t the only language that Julia can call upon using a built-in package. It also offers R integration via Rcall, and can import C libraries with ccall. This makes it a far more general purpose language environment, and could tempt over users from those languages looking to plug a gap in their own syntax or improve performance.

Use cases

So, what is Julia good for? It comes from a numerical computing background, and a lot of its package development has focused on that. One place where it excels is in plotting. Packages like Plots.jl produce pleasing data visualizations, and several packages can animate plots in realtime. On the data science side, the DataFrames package takes users well beyond the basic data structures in Julia, and there are packages that let it integrate with databases including Apache Spark.

Machine learning is another area where the Julia community has made strides. It is GPU-friendly and features its own CUDA stack for interacting with NVIDIA kit.  

So, scientific heavy lifting are Julia’s mainstays. But what about if you wanted to just build a web app with it? The community has helped out with general purpose computing, too.  Genie is a full-stack MVC framework for web apps that is based on Julia, and for those that just want to build user interfaces, there’s Escher and WebIO.

Julia may still be in its infancy compared to heavyweights like Python, Java and C. Nevertheless, it’s a pleasant, easy to use language with solid performance and a strong and vibrant community. The YouTube video channel alone shows you how committed the team is to the cause, and Julia Computing got $4.6m in seed funding 2017, which gives it some fuel to take things to the next level. It’s worth playing around with.