Stevan Little is the author of Moose, the library introducing Perl 6-inspired classes in Perl 5. He also started the p5-mop project, which was aimed to bring classes to the Perl 5’s core.
Perl and OOP
Once you said that it was Damian Conway’s “Object Oriented Perl” book, which gave you the idea of real power of Perl. Why did it happen with Perl while other languages offered much more developed OOP?
This was one of the books I read when I was first learning Perl, and the truth is that I have a love/hate relationship with the second half of that book.
The first half of the book is a great introduction to Perl 5 OOP and full of practical information and advice for professional Perl programmers. But I have long suspected that the second half of that book was written during an intense hallucinatory fever dream caused from a bite by one of the many poisonous creatures that inhabit Damian’s native Australia, … I can see no other plausible explanation for it.
Let me provide a quick bit of background, before I actually learned Perl I had been mostly doing Javascript professionally, which also has a quirky but flexible object model. In my spare time I had also been exploring several other languages and had fallen in love with LISP, and the idea of a multi-paradigm meta-circular language. I had also been exploring Python and Ruby around the same time (early 2002) but their more (as you put it) developed OOP felt restrictive to me, and to me Java/C++/C# felt downright boring. So when I actually read this book, and in particular the second half, I saw in Perl the same power I had seen in LISP which for me was very exciting, … this is the “love” part of the relationship.
The “hate” part of the relationship came when I first started to read other people’s Perl code and saw people actually using the crazy techniques that Damian described. While it is very interesting to be able to do things like strong encapsulation through blessing closures, it is kind of an esoteric technique and not really something any sane person would recommend for production code.
I suppose I have the same love/hate relationship with Perl itself. Perl is an extremely flexible language and amazing things can be done with that flexibility, but never without a cost. Perl will let you do things completely crazy things and make no moral judgement as to whether you should do it or not.
In many ways, this is the antithesis of the more “manager oriented” languages like Java, in which many of the language features are there to help encapsulate bad implementations behind controlled interfaces. The explorer in me loves Perl for this, but the manager in me hates Perl for this.
In Perl 6, the OOP features are clearly documented and available without external modules. Does the TIMTOWTDI slogan still work with classes in Perl 6?
This is a very good question, and honestly I am not so sure that it does work in the same way for classes in Perl 6. However I think that this is a very good thing and I see it as an evolution of the concepts that TIMTOWTDI embodies.
I believe TIMTOWTDI to be a key ingredient in making Perl feel fun and creative, but in relation to OO in Perl 5 it manifests itself as a bunch of largely incompatible approaches. This is really not a useful feature for day to day programmers and can even become a community wide distraction.
By introducing an extremely flexible “One True Way To Do OOP” I think Perl 6 ultimately will allow Perl programmers to be able to focus their creativity at a higher level. It is my hope that in Perl 6, that same creative energy will be applied to how OOP is “used” and not so much how it is implemented.
Pugs
What did you do when you were working on the Pugs project?
I spent a lot of time working on the test suite at first, specifically trying to get it organized well so that we could detect regressions more easily but still retain the development velocity. I eventually got involved with implementing the Perl 6 object system, I don’t recall how or why, but that ultimately became my focus in the project.
There were several approaches to implementing object system in Pugs. What were they?
Well, the original design was by Sam Vilain in Haskell, but this was really just a theoretical exploration that Sam and Audrey were playing around with. At the time, the Pugs interpreter was not sophisticated enough yet to support OO features.
When I got involved, I started by reading a bunch of books/articles/papers about the object models of Smalltalk, LISP (CLOS), etc. and then exploring these different approaches in Perl 5 prototypes. The real turning point for me came when I read a book called “The Art of the MetaObject Protocol” which was about CLOS, the Common Lisp Object System, and happened to include a very readable reference implementation. It was through this book that I really finally grasped the idea of the MOP, which is really just a fancy way of saying “an API for Classes, Methods, Attributes and Instance”.
The most complete prototype was the Perl6::MetaModel module, it was meta-circular and involved a complex multi-stage bootstrapping in which each stage introduced another element of the MOP so that the subsequent step would be able to use it. It was at this point that Yuval (nothingmuch) Kogman also got involved in the work, which lead to the design of the MO system. With MO, Yuval expanded on many of the ideas from the prototypes, but instead of just mimicking functionality in a host language he designed something that was much more suitable for the internals of a compiler. Unfortunately the Pugs project lost momentum shortly after this and MO never got implemented beyond a few basic prototypes.
What did you miss in early prototypes?
I am sure we missed a lot of things, Perl 6 was still heavily in the language design phase and the meta-model was something that was only alluded to in the Synopsis.
Besides Yuval’s MO, none of the prototypes I wrote were appropriate for inclusion into the language itself, they were all just mimicking the functionality of Perl 6 OO within the prototype host language (Perl 5). We did begin exploring this towards the end of the Pugs project, but nothing was ever completed.
As part of the Pugs team, what do you think of the -Ofun method?
I think -Ofun was the correct approach for an exploratory project like Pugs, which is why I also adopted it for my Moe project. I think it is important to be able to prototype and explore concepts without the pressure of having to produce a production grade system, the concept of -Ofun I think really encapsulates that idea very well.
Moose
Why did you want to have classes in Perl 5 if Perl 6 was already on its way?
Because I am impatient 🙂
Moose was first released in early 2006, and while I knew Perl 6 would eventually come out, I wasn’t sure when and I really wanted to have some of the nice features of Perl 6 in my Perl 5 codebase.
Why does Moose require the “half of CPAN”?
Unfortunately, Moose’s dependency list has grown over the years, but the source of this meme was actually the fact that we have always insisted upon a reasonably up to date set of Test::* modules. If your copy of perl was not up to date, this caused an explosion of dependencies to be pulled in, which of course resulted in very long install times because CPAN.pm can be a pain like that.
If you actually take a look at Moose’s dependency list many of the modules are fixes or improvements on some core Perl feature, some of which have since been folded into the core. The best example of this is Devel::GlobalDestruction which was written to expose perl’s internal global destruction flag, then in perl 5.14, this was added into core. Another example is Sub::Name and Sub::Identify, which are used to get and set the internal name of a subroutine, in the recently released perl 5.22 these functions have been added to the Sub::Util module which is part of the core Scalar::Util module.
It always baffled me that people complained about this, the time savings that Moose provided programmers in their day to day work easily trumped the time loss of having to sit through a long install process which is something you do much less often.
When designing Moose, did you think that Perl 6 will ever exist?
Yes, I was counting on it. I always saw Moose as a way to potentially bridge the gap between Perl 5 OO and Perl 6 OO, sort of a gateway drug to Perl 6. I will admit to having some doubts about Perl 6, but I knew it would eventually happen because I had seen first hand the passion that was behind it.
But of course this didn’t help existing Perl 5 programmers, which was my goal with Moose. Even today legacy Perl 5 codebases exist and can not easily be just ported to Perl 6, we have only to look at Python to see how difficult it is to introduce a new version that is not fully backwards compatible. Moose always aimed to work well with existing Perl 5 OO code and support what I called “Partial Moosification”, meaning that Moose did not require you to port all your code to use Moose, you could just convert as much as needed and the rest would still work as before.
Moe
What is Moe?
Moe was basically something I had wanted to do for a long time, which is to write a language interpreter for a Perl-like language.
The original motivation came after my initial attempt at adding classes to Perl 5 (a project called p5-mop) had failed to prove fruitful. But it was further bolstered by some of bikesheding discussions I had been seeing on the perl-5-porters mailing list surrounding Peter Martini’s work on subroutine signatures. I wanted a project that would be fun, interesting and challenging and which was going to be -Ofun from the start, very much in the spirit of Pugs. Unfortunately I did not succeed in attracting the level of contributors that Pugs did (not even close), but that was largely due the fact I was having so much fun coding that I forgot to blog about it.
I actually ended up putting this project aside to go back to working on the p5-mop project. In a number of ways I think this work was helpful in giving me a better perspective on the internals of a language compiler/interpreter without having to also learn C, XS and the Perl core at the same time.
Moe is a “prototype of Ultra Modern Perl 5”. What is Ultra Modern Perl 5? Also, comparing with Perl 6.
Well, I chose “Ultra Modern” mostly because it sounded cool, but also Moose had been dubbed a “Postmodern Object System”, so why not keep with the whole modernity theme.
Ever since the Pugs project I had a vision of a language that was somewhere between Perl 5 and Perl 6, a Perl 5 ½ if you will. The idea was to basically borrow a lot from Perl 6, but leave out some of the more exotic features like grammars, hyper-operators, etc. and to explore creating a new runtime which had desirable features like an AST, an OO runtime API, a built in MOP, and more.
When you select features for Moe, how do you choose between the Perl 5 and Perl 6 approaches?
I had already decided on the OO features I wanted via the p5-mop project. I also wanted the built-in functions and data structures to be object oriented, so we borrowed some of the Perl 6 core class hierarchy design for that. I tried to strike a balance between Perl 5 and Perl 6 syntax, however that ultimately ended up being more Perl 6 then Perl 5. In the end, it became less of a conscious decision and more just a matter of testing out the different ergonomics of both and making a decision on which felt best.
I actually found this to be one of the most interesting parts of the project, getting the interpreter to a stage where I could really start to “feel” the language and then being able to tweak it and experiment with ideas.
Taking into account that you dropped the most awkward Perl 5 elements in Moe, is creating the Perl 6 compiler more difficult than the one for Perl 5?
They are both insanely difficult things to do, insane in different ways, but equally insane.
As I explore the Perl 5 core more and more I am both impressed and horrified at some of the twisty mazes of C code that exist in the core simply just to make Perl 5 behave in that “intuitive” way many of us have come to know and love (and sometimes hate).
At the same time, one of the most impressive and horrific things about the design of the Perl 6 language is that Larry and Co. have managed to take many of these features and behaviours, formalise them in a spec and implement them in a clean object oriented way on a MOP powered VM.
The languages in the Perl family are really optimised for the humans that write code and not the computers that execute it. The consequence of this design decision being that the compilers and runtimes are necessarily complex in some very weird ways.
Why did you choose Perl 5? Why Moe is not an implementation of Perl 6?
Someone was already working on Perl 6 🙂
By the way, Audrey Tang mentioned that Scala could be an excellent choice for a Perl 6 compiler.
Yes, actually Audrey and I spent a lot of time looking into it during the Pugs project and Scala’s implementation of traits influenced my work on Roles, both in Pugs and later in Moose.
However the real reason has more to do with the fact that I had wanted to use the JVM and Java’s (comparably primative) type system kept getting in my way, whereas with Scala, things just flowed. I would attribute this to Scala’s functional programming influences as FP language tend to be very nice for writing compilers and interpreters in.
I actually have been giving some thought to the idea of re-writing Moe using Perl 6, we will see if that ever happens 🙂
Is it possible to change the grammar and use Moe’s skeleton for building a Perl 6 compiler?
It is possible, but so many other changes would be required in the AST and Runtime that it wouldn’t be worth the effort. Moe was a playground for ideas, while I am happy with some of the design it still could use a lot of work and refinement.
Can Moe produce output that can be run on MoarVM, for example?
Yes, … in theory. The parser produces an AST which the interpreter then walks to execute the program. It should be possible to take that same AST and have it compile into MoarVM bytecode. However I say “in theory” because I was designing the AST specifically for the interpreter and I suspect that it was relying on the runtime system too heavily, it would need some re-working to make it useful to a compiler.
Perl 5 with Classes
What’s the difficulty in embedding “normal” classes in Perl 5?
The biggest difficulty is actually not technical, but instead just the act of trying to add a new feature to an old language, I grossly underestimated the complexity of that task.
The biggest technical difficulty was that in order to do this right, I needed to read and understand the Perl core.
When I first approached the idea of adding classes to Perl 5, I tried very much to build a completely enclosed system that would have minimal contact with the existing core language (for all sorts of reasons). I basically did not want to get my hands dirty with the notoriously ugly and messy perl guts. In the end, I think one of the key reasons why I consider several of the p5-mop prototypes failures is that this approach is just completely wrong. It has taken me several years and a lot of reading through the perl source, but my current approach is to “embrace the legacy” and really try to improve upon Perl 5 OO instead of try and replace it.
The best part actually about this approach is that I have managed to “implement” a number of key features without writing any code at all. If you look hard enough and dig deep enough, there are lot of features in Perl that can be repurposed in a new context to implement functionality that is needed by a MOP.
You work on the p5-mop project, which should make classes available in Perl 5. How do you balance (or persuade others) between backward compatibility and the need of having new syntactic features in Perl 5?
This is something I have struggled a lot with over the last few years, my earlier prototypes went too far. Then I saw Peter Martini’s talk at YAPC::NA 2013 in Austin in which he discusses the process he went through in designing the subroutine signatures. One of the key design goals he had was that it would add the absolute minimum to the language, both for the sake of core code maintainability and language backwards compatibility. I have adopted this approach myself, but it is a slow process since it really does require an understanding of the Perl core codebase.
Perl 5 vs. Perl 6
Is it worth improving Perl 5 instead of developing Perl 6?
Why does it have to be just one and not both? I believe that Perl 5 the language still has life in it, but only if it can continue to evolve and improve. At the same time, in order for Perl 6 to gain traction users will need to start developing modules and building up CP6AN.
Lots of Perl 6’s features are already in Perl 5. After p5-mop is in the core, we will have classes too. Do we need Perl 6 then?
Yes, because Perl 5 will never get things like grammars, proper concurrency or a type system.
What should force developers considering migrating to Perl 6 if the difference between the languages will be less significant?
I believe that Perl 6 has many other compelling features to offer (grammars, concurrency), and it will be those features that will dictate the decision in to use it or not.
Do you see any perspectives of cross-languages sharing between Perl 5 and Perl 6?
Yes, and it is my hope that getting classes into Perl 5 will help this effort, which is one of the reasons I have been focusing a lot of the C part of the p5-mop API.
Future
What else could Perl 6 borrow from other languages?
I think Perl 6 has enough features actually 🙂
Will you switch to Perl 6 when it is ready?
Perl 6 is already part of my toolset and I have plans to write a few modules in it when I next have some free time.
I actually think it is important that people not wait for Perl 6 to reach some mythical ideal of “finished”, but instead to start using it now and help the implementers to further refine both the compiler and the language.
How do you think, will Perl 6 be less “read-only language”?
No, but that is mostly because I don’t think that Perl 5 is a “read only” language. It is entirely possible to create untold horrors in any language, Perl 5 is really good at this, but it is a choice, just like any other.
Will the users create non-maintainable dialects?
Actually, that has always been my biggest worry with Perl 6, while in other aspects I think it improved on TIMTOWTDI, the pandora’s box that dialects will open up could be a real nightmare of TIMTOWTDI gone bad.
Is the “design in public” method good for designing a programming language (and its implementation)?
I am not sure that question is answered yet, we have to wait to see how Perl 6 and its implementation does to really know for sure.
One thought on “Interview with Stevan Little”