Summary🔗

• Dr. Odersky's talk "Simply Scala" connects language design and coding tactics with developer onboarding and retention.
• Despite many completing Scala courses, there's an industry narrative that hiring Scala developers is challenging.
• Dr. Odersky suggests simplifying Scala and developing a "blessed" set of libraries to aid retention.
• Haskell's history provides insights: attempts at simplifying or creating platforms can miss their mark or have unintended outcomes.
• Onboarding: mastery of a language involves not only its syntax and semantics, but also its industrial application.
  • Thus: provide beginners with a menu of different solutions to a similar set of problems within a domain, explaining which solution is appropriate when.
• Retention: incentive for users to use one language or another comes by and large from the business landscape.
  • Focusing merely on simplicity risks missing out on the advanced uses of a functional language, limiting its business applicability.
  • Thus: instead of mere simplification, functional languages should strive for innovation that capitalizes on their foundational strengths.

Simply Scala?🔗

This post has stemmed from me pondering on a great talk by Dr. Odersky called "Simply Scala".

In this talk Dr. Odersky links programming language design and tactics used in software development with the matters of developer onboarding and retention. He lists some courses in which people are known to learn Scala. Namely, his Coursera course and JVM Rocks masterclass. He then mentions that despite people demonstrably completing these courses en mass, the industry still complaints that it is hard to hire Scala developers.

Interleaved with this, he analyses some of the more confusing bits of Scala, advocating for:

• Smart use of mutability.
• Less reliance on categorical type classes.
• More reliance on in-built language features.
• Knowing when to let go of type safety and replace it with pre- and post-conditions and testing.
• Developing an opinionated set of standard approaches to the standard challenges.
• And more, see the talk, it's really good and useful!

Finally, he makes a judgement call that the onboarding and retention problems must stem from the issues with the simplicity of the language and the exact issues with simplicity of the tactics used in the large codebases. Thus, he proposes two big ideas:

1. Platforming of the languge: making a "blessed" set of simple libraries of the industrial quality.
2. Simplifying the language: discovering and enforcing ideomatic approaches as well as reducing surprises for beginners.

I wholeheartedly agree with the practical advice Dr. Odersky gives when it comes to programming in general and functional programming in particular. However, I disagree with the implication that the reasons for the problematic onboarding and retention is rooted in these issues. See, I can't shake off that nagging feeling that I've seen people relating "simplicity" with onboarding and retention before, in Haskell.

Cautionary Tale of Many Haskell Platforms🔗

Now we wait for GHCUp to bite the dust.

In the late 2000s and early 2010s, Haskell community faced a problem that its package repository, hackage, hosted a lot of mutually incompatible packages. It created a problem called "cabal hell" by the practicioners. Because cabal (version one) was the build tool for Haskell, which has ambitiously tried and failed to also be a Haskell package manager.

Then there were two big-picture approaches to solving this issue, one was something I call "pragmaticist" and the other was something I call "platformist". Pragmatics tried to solve the issue by various means, respecting the pluralism of software and libraries published. Two popular approaches was to use Nix to build Haskell packages and abolish the package management capabilities. This approach was somewhat quietly used for Haskell software written and used in CERN since before 2013.

Another approach was to create curated sets of mutually-compatible packages which Michael Snoyman et al. have released and popularised in 2015.

In contrast, around 2010, platformists have created a "batteries included" Haskell installer which worked great! Until the batteries run out, that is. Meaning, until you needed to do something that wasn't included or thought about by the "Haskell Platform". You would think that Stackage also suffered from this issue, but actually not. It was trivial to add your own or third party libraries to stackage by forking or overlaying and the whole product was hinging on a single and simple standard of package definitions.

What that meant beyond openness, was that Stackage was also interoperable with other pragmatic solutions, like Nix. Just like Nix was interoperable with Stackage! Feel free to check out my edgy slides about stackage and using nix for haskell. Fun times.

Needless to say that, in my opinion, all the work done by very intelligent and capable "platformists" would have had a tenfold impact if they would contribute exclusively to pragmatic tools. That's why when I hear the ideas like shoving sbt under the rug because it's "too complex" together with the idea to make a "Scala Platform", I can't help but to remind of a similar trajectory taken by part of the Haskell community.

Dangers of Absolute Primitivism🔗

Another word of caution I would like to say when it comes to the idea of simplifying the language, even though I think that Dr. Odersky is on point with his vision for simplification, is what I call "absolute primitivism". This, again, stems from the history of Haskell and is related to a now inactive, but nonetheless still detrimental. Sadly, the peak of its popularity coincided with one the more active phases of development of DependentHaskell and, since GHC (the Haskell compiler) is governed by a steering committee, the pushback from the vocal minority has significantly reduced the traction and support for the dependent Haskell initiative.

And please don't let the irony of this be lost on you. Most of the Haskell libraries used in the industry rely heavily on the "advanced types". For example, "servant", the most popular API framework, provides immense compile-time guarantee and API client code generation leveraging type-level programming.

I don't think that it will be a big problem for Scala, as the language's philosophy is as far from absolutism as it gets while the governance, to my knowledge, is more centralised. Nonetheless, it's important to remember the lessons of "Simple Haskell".

So What to Do?🔗

As you could have guessed, I don't think that attributing perceived programming language and codebase complexity to onboarding and retention is correct. Furthermore, I don't think that onboarding and retention is connected at all. Even the talk by Dr. Odersky itself hints at it when he mentions how many people complete the courses (onboarding) vs. how undersaturated the market is with Scala developers (retention). What the "platform and simplify" policies imply here is that onboarding is a two-step process:

1. Learn the language itself.
2. Learn how the language is used [in the industry].

All of us at ∅hr share the opinion that the way a language is used industrially is part of learning the language. That's one of the reason our test tasks are always modelled after the real industrial code bases and are exactly the same regardless of the seniority the candidate is applying to.

Thus, in my view, onboarding and retention should be addressed as two separate issues. For now let's focus on onboarding.

Dr. Odersky has brilliantly demonstrated how to use an appropriate level of abstractions in his talk while talking about the post-conditions in AST processing. It's about finding the trade-offs. But even appreciating the spectrum of possible trade-offs requires pattern-recognition and experience.

Thus, I think, we shouldn't just limit the tools available to beginners. We should instead introduce the spectrum of possible trade-offs by showing them a set of code samples that address different needs in a similar problem domain! That's how they will get the lay of the land when it comes to abstractions and will learn to recognise the reasons why some may be used. Free bonus — they will also have some idea about the justification needed for abstraction use. Know from the get go that learning a new cool trick is not a reason for them to apply it to everything.

It follows that the idea of "platforming a language" or "simplifying a language" doesn't address any of the actual simplicity of use issues. The only exception would be the simplifications that streamline the ergonomics. But as I mentioned before, I don't think that the issue with beginner adoption and continued use is actually the issue in ergonomics. It's by an large an educational issue rather than just a design one.

...And Hiring Existing Functional Programmers?🔗

The second issue formulated as "finding XY developers is difficult" is all about retention of developers in technology XY.

In the previous decade, I would just brush it off with a claim that it's just that HR doesn't understand where functional programmers hang out, how they think and what they need.

But I no longer believe it's that simple. We are almost in 2024 and we have to understand that there are two winning languages: Rust and TypeScript. These languages have taken the best out of "our" languages and leveraged things borrowed (some pun intended) in the domain of systems programming and web programming repectively. It is very difficult to convince intermediate developers to keep using the "old guard" languages when a lot of static guarantees can be replicated in the two most growing languages today. And this, and I can't stress this enough, is another reason why simplifying is a losing proposition! We can't impress anyone anymore with claims that solid immutability support, lazieness and tight typing can reduce or eradicate whole classes of runtime errors. Of course, many practicioners still lag behind functional programmers in terms of their understanding of how to apply typing and immutability to their code. They also may not know that classic immutable data structures, modern immutable data structures and postmodern immutable data structures can be "blazingly fast". But both PLT people from the imperative world and the business is on board. Teach someone how to Haskell or Scala and they will simply Rust better. "Our" languages need language features that are if not novel, then at least novelly integrated into a single language system.

Don't get me right, I'm not suggesting we drop everything and start porting ATS compiler one to one as Scala 4, but if I was to start an FP-consultancy in 2023 (and I know a thing or two about running successful FP-consultancies), I wouldn't bother with Haskell, until dependent Haskell happens and even then. I would take Lean 4 in which I've written far-from-trivial codebases, and then leveraged the actual market differentiator of inbuilt theorem proving, dependent typing in presence of very efficient compiler and runtime. I sincerely think that we can'd analyse the business state of functional programming by looking at the exodous of programmers and money from our technologies just in our own echo-chamber. It's a massive red herring.

Conclusion🔗

Dr. Odersky's examination of Scala's complexities and the suggested reforms, such as platforming the language and embracing simpler idiomatic practices, highlights a broader concern: striking the right balance between a language's power and its appeal to new learners and practicioners. We have examined Haskell's historical foray into platformism, which offered significant lessons. However it's crucial to remember that challenges with onboarding and retention don't always stem from the language's structure alone. The industrial application of a language is also a key component of mastering it. Onboarding strategies should evolve to educate beginners about the nuances of trade-offs and abstractions, not merely limit the tools available.

The technological landscape has shifted with languages like Rust and TypeScript basing their paradigms on functional principles and many more melding FP into their DNA. Traditional functional programming languages need to keep innovating to stay ahead of the curve. The quest shouldn't be just about simplification, quite the opposite. It is about offering businesses and developers features and practices that further elaborate on the historic success of functional languages.