CMOS #2: Sawyer X on Perl 5
Interview with Sawyer X, the current Pumpkin of Perl 5 about the Perl 5 Porters and the programming language in general. He also explains the difference between Amsterdam.pm and AmsterdamX.pm.
This is Gabor Szabo from the Code-Maven podcast and screencast and video interviews. With me is Sawyer who is current Pumpkin of Perl 5 and we are going to talk about Perl, the language and the compiler.
I started learning the language which the basics was fun. I've learned it in one night, like at 4 am, it was a lot of fun. Then I've started writing some code, doing different stuff. I think the first thing I wrote was actually a program that downloaded all of the comics of a certain comics strip that I liked, and put them in nice directories, because the Internet was not very fast and I wanted to download all of them and then read them at home.
I've started attending the Tel Aviv Perl Mongers group, TelAviv.pm, which you organized. I gave a few talks there and eventually someone, you, convinced me to go to FOSDEM and YAPC::EU, where FOSDEM is the Open Source conference in Belgium, and YAPC is the European Perl Conference, and you suggested I give a talk there.
I remember we had a discussion about this where you said "Go ahead" and I said "No way" and you said "go ahead" and I said "no way", and we did it back-and-forth a few times, until I said "fine, I'll go whatever".
It was a lot of fun and I just kept giving talks and I've started to be more and more active in different projects. I became mainly active in the Dancer web framework which I started using for a personal project and slowly I had more and more interest in it, and became the maintainer of it, and we grew a nice community around that.
When I moved to The Netherlands where I live right now to work at Booking.com I created another group. We already had Amsterdam.pm which is a Dutch Perl Mongers group and I founded the AmsterdamX.pm which is not X for Sawyer X, but it is X for expat. I really have to stress that. AmsterdamE is not as AmsterdamX for expats and the idea is that it is much more comfortable for people who don't speak Dutch and the timing is different and it is a bit different. We found it to be more successful because we had a little participation from expats. We have a large group of people doing Perl in Amsterdam and in the area but they would not go to the Perl Monger's meetings so I started that group.
This year Ricardo Signes who was the Perl Pumpkin for the last few year stepped down from that role and I took over it. Mainly passed on from him, less of a "I fought him for it," and that takes most of my time nowedays. I still do a lot of conferences. This month I think I'll be at 3 conferences and along with that Dancer which I am still active in and, of course, work. That's basically my time nowadays.
The Perl Pumpkin. I don't know if I have a good story for it. I was always interested in internals of Perl, it is written in C and.
I was always interested in the internals. I came from C which is what Perl is written in. It is very sophisticated C and I never got to those levels, but I was always interested in that I was always trying to keep track of what's going on in the language itself. You know, the underpinning of what we know as the Perl syntax and how that works and optimizations there. Lexing, parsing all that stuff is very fascinating but the mailing list for that is just too much.
I don't know if you have ever read the Perl 5 Porters mailing list. The group that maintains Perl is the Perl 5 porters and the name P5P is the name of either the group or the list, of course.
The list is very hard to follow because it is basically an amalgamation of several different communication channels. So there is the tickets, and there is the discussion, and then there are some summaries that are sent there for some commits. At least there used to be. It could be a lot of different stuff and they could be all combined into one list, which is what P5P reads and writes to. It was just too much to keep track.
So I started summarizing them. So now I read them every day and then at the end of the week or at the beginning of the week I publish a summary of it that you can read on the list or on a different list that's called perl 5 summary, that's just for summaries, or on Twitter and also on blogs.perl.org.
So I became active in the core group mainly following discussions. Giving notes here and there, getting more interested in the internals. I was always very active in the community prospective of Perl and I was always wondering about the direction and the vision of Perl and where to move forward with it. Specifically Perl 5.
So when Ricardo decided to step down, he was looking at what kind of people can fill the void, and could take on this role. I was nominated by someone from the core developers. It was discussed internally and a decision was made that I'd do it appropriately, which hopefully I would. So basically what it means...
Once in a while there are conflicting ideas, one person implements things, I think it should be this way, and another person says, it should be done that way, and there they just reach a point where they just disagree and they cannot resolve it. Then I come in and I make a decision, Okay, we are taking this approach because of A, B, and C. The thing is that I don't know all of the technical details that a lot of the bright people there do know, so I mostly listen a lot and then talk to people, oftentimes off the list to get more information, and then I can make decisions.
So actually a lot of my work is not done on the mailing list. You won't see me as much as other people, mostly I talk to people outside the mailing list to get a better understanding of topics before I respond. So my response is usually at the end saying, "Yes." Or I don't know, what about this direction? So that's kind what the Perl Pumpkin does.
I think it would the biggest value of it. It would be hard to run Perl code without it, but, I mean there are a few other interpreters for Perl 5, and you could use them. They are either forks or they have limited capability in comparision, so the cannonical implementation of Perl 5, which has the larges group of developers that are committed to it, including grant money and sponsorhip money, is Perl 5 Porters and what we call Perl.
But jokes aside, Perl is a language that can solve a lot of different problems. It has a multi-paradigm programming perspective. So you can writen in procedural, you can write in Object Oriented, you can write in purely functional, you can write in so many different ways, so you can apply it to so many different technologies.
Because it has a good coupling with C, using the XS layer and FFI if you want it, it allows you to interact with C or with C++ code fairly easily and interact with Perl. So you can speed up parts of it or combine with libraries etc. So it is very high level. It is hyper-dynamic, which is a term that I really enjoy using for that perspective, and it has a lot of really cool features under the hood.
I use it to solve most of the problems that I have.
The regular expression has a strict mode. Unicode 7 was introduced and added last year. This year 5.24 already has Unicode 8, and we've already merged Unicode 9, so we'll release it in the next version, which will be 5.26. Aliasing by references for example. Having const as a subroutine attribute. There are a lot of stuff like that. If you have a look at the recent version for example that just released recently. It was released in May 9th, 2016 there are a few other stuff. So there is Unicode 8 I've already told you about.
There is also postfix dereferencing. Once of the things pissing people off, when you have a very big structure that represents an array, or it can also represent a hash. You have to write the entire structure all the way to the last element and then you have to wrap it with a dereferencing in order to got what it references, which is kind of using a pointer and dereferencing a pointer. What happens is that you write all of this forward and then you have to wrap the entire thing. Because that's how the wrapping code works. But now we have added syntax to change that to the end. These are things that seem simple, but it completely changes the way you write and the way you work, because you only move forward. Which is some of the things people like about functional programming. The way that you move. So these actually have a lot of value and this is no longer experimental. You can use this. Things like optimizing basic operations. Additions, substractions, multiplications, subroutine calls, you know stuff that people tend to use. All of these have been optimized significantly.
We are introducing the indented heredocs synatax, a bunch of other languages have it. Now we'll have that too.
A lot of security-related changes. For example Perl developers familiar the current directory being in what you load when you try to load modules. It also tries the current directory. We are going to remove that. That's a security issue. While people have used it, it is actually very problematic. We realized this and it is going to change in the next version.
If your code assumes loading from the current directory, you can just say I'd like from the current directory. That would work just fine. Any code that does that will work just fine. But code that implicitely loaded from the current directory, that is going to be removed. That is a much safer perspecive of this.
So if you take a few steps back and you look at all this history. What we have added was a lot of syntax, a lot of changes in bug fixex, security, a lot of optimizations. We have plugable keywords so you can write an interpreter within the interpreter. Really crazy stuff like that. These are things that keep evolving and changing in the language and you can't really see all of this unless you look at the global picture. The whole picture of it.
Modern Perl tends to be more succint, but it is very good and Beginning Perl is very thorough and I think Curtis has a very good way of explaining things and they both teach you, not Perl like 10-20 years ago, but Perl that you'd write nowadays, that people would expect you to write, that is proper practices that don't assume on "I know Unix" or don't assume "I came from C" and it is exactly the same. give you a very idiomatic and correct way of writing Perl that is extensible and secure and correct and fun. So I really recommend those, if you'd like to learn the language.
When it comes to how to improve the language itself. So the language breaks down to two parts. It breaks down to the core of the language and all of the modules for it. The modules for it are written in either C, using a binding loader called XS. It's kind of a glue language between C and Perl, or directly in Perl which we call Pure Perl because it does not have any XS or C. And you can write a module that is either in one of them or in both, whatever you want. So a lot of the language extensions are done using modules and the language tries to have a small core that is easily extendable using modules. So a lot of things that we assume as the core language are actually just modules that you can open a pull-request for you can contribute you can work on these projects openly and it's a lot of fun and there are project. Examples is the one that I am involved with is Dancer that you could contribute code to and you can join those teams. Usually it is a team of people. Sometimes it is a single maintainer. Sometime is one or two maintainers working together and you can always jump in and say hey, I know a little bit of Perl I want to help with this.
There is also the CPAN Pull-request challenge. The Pull-request challange kind of allows you to get a taste of it, like a tasting menu where every month you get a new module and this module was curated to be both important and used and having issues to work on. So there is a filtering process going on there. Not just random stuff, because CPAN can take whatever anyone wants to upload. CPAN is where we upload stuff so it contains anything. Some of those no one really uses. Some of those are not very good, but some are very important and used and they get curated and you get every month "hey here is a thing you could work on". You can go and submit a pull-request and talk to the author and it is fantastic.
Now when it comes to language itself, if you'd like to work on the language, that requires... I mean at first I though that it requires learning C like if you want to learn on the language then you need to know C. In fact the matter is that the language contains a lot of tooling around it that are part of the language that could use a lot of help. Many of these tools have been written in Perl. I think the first contribution that I did was remove a core module and put it on CPAN. One additional thing that we have now is that we have another module that we are thinking of making available separately from core as well. Because that would allow people to use it without necessarily having to upgrade Perl. That is a contribution I can make knowing any level of C. So I've been trying to collect these nowadays. Haven't got very far but I am trying to collect all of these different things that you can do that don't necessarily would require specific C knowledge. Now if you do know C you can always take a look at the existing tickets in rt.perl.org and if there is any ticket there that you want to play with the Perl source is open sourced fully. Free software just clone it from Git. It's also available on a GitHub mirror, an unofficial GitHub mirror. It's available there. You can just submit patches, you can e-mail the mailing list Perl 5 Porters and you can contact us and say "Hey I'd like to work on something, what do you work on?". Or "I'd like to work on this ticket, what can I do?". You can also comment on the tickets themselves. So that ticket and the whole combination of the mailing list, if you comment on the ticket we'll see it and comment onthe ticket back. If you write on the mailing list, everyone else will see it as well. So it all gets combined. So no matter how you access us, no matter how you contact us we'll find it.
And Zefram who is an expert on incredibly detailed technical topics.
I want to thank Jim Keenan who is a fantastic curator of all the tickes. If you open a ticke he will see it and if you have a test he'll run it and if you have a patch there he'll try to merge it and if its wrong. If it does not pass any kind of policy that we have, then he'll comment with explanations and links. It's just incredible.
Todd Rinaldo and John Lightsey who worked on the changes to the current directory. They work in the security team at cPanel. And cPanel has like an entire security team that does stuff. They found a slew of these issues and they raised them, they wrote the patches for them. Just fantastic work. Matthew Horsfall who is putting the indented heredocs. I mean there is a list. I am not gonna be able. I mean if you'r going to have another episode we can sit down and just go.