WEBVTT 00:00.000 --> 00:09.000 So welcome everyone to Lighting Talks. 00:09.000 --> 00:15.000 We're continuing with Flavio, which is an amazing project for running Pearl. 00:15.000 --> 00:17.000 Okay. 00:17.000 --> 00:20.000 Thank you for coming. 00:20.000 --> 00:25.000 So, yeah, this is about Pearl Programming Language. 00:26.000 --> 00:32.000 It was developed in the 90s to get a bit of internet. 00:32.000 --> 00:40.000 And Pearl was basically created on top of seed language, 00:40.000 --> 00:44.000 a way to make things easier, 00:44.000 --> 00:49.000 parted to many different computers. 00:49.000 --> 00:54.000 So basically Pearl ran everywhere that seed language was available. 00:54.000 --> 01:05.000 And when the internet came, Pearl was the easiest way to put things online. 01:05.000 --> 01:10.000 So it got a really popular at that time. 01:10.000 --> 01:14.000 So this is the Pearl Port page. 01:14.000 --> 01:17.000 It shows where Pearl currently runs. 01:17.000 --> 01:21.000 And you can see a long list of unix derivatives, 01:21.000 --> 01:30.000 does even windows and really some less popular computer computers. 01:30.000 --> 01:40.000 Yeah, the MS and things we probably don't have access to, 01:40.000 --> 01:45.000 like ABCD platforms, mainframes. 01:45.000 --> 01:52.000 So it's a bit of surprise that Pearl doesn't run in the JVM. 01:52.000 --> 01:59.000 So why not, is several attempts were made, 01:59.000 --> 02:05.000 but traditionally Pearl runs on top of seed language. 02:05.000 --> 02:14.000 And yeah, running seed on top of JVM is actually making it very slow. 02:15.000 --> 02:21.000 Pearl has its own VM, its bytecode interpreter. 02:21.000 --> 02:29.000 And parting VM to run on top of another VM makes it, 02:29.000 --> 02:33.000 yeah, consider that you have the raw CPU, 02:33.000 --> 02:36.000 a VM makes it maybe two, three times slower. 02:36.000 --> 02:40.000 If you create another VM on top of it, it would be ten times slower. 02:40.000 --> 02:47.000 So that's really, yeah, not going to be practical. 02:47.000 --> 02:51.000 But yeah, running Pearl without seed means rewriting. 02:51.000 --> 02:55.000 And there's a lot of problems with that, 02:55.000 --> 03:01.000 especially the semantics, what help our works. 03:01.000 --> 03:06.000 It's a very fine tuning of features. 03:07.000 --> 03:13.000 It's very hard to describe this in a different language from scratch. 03:13.000 --> 03:22.000 So what makes this project possible is that, yeah, AI. 03:22.000 --> 03:30.000 I made a small, so I figured I could make this, but with help. 03:30.000 --> 03:35.000 I wrote a small prototype in Pearl and Java, 03:35.000 --> 03:39.000 to figure out if the concepts would work, 03:39.000 --> 03:43.000 like closures, the parser, 03:43.000 --> 03:48.000 and they figured way to make it lookable, 03:48.000 --> 03:53.000 like a very small parser with sub languages. 03:53.000 --> 03:58.000 And yeah, it actually worked. 03:58.000 --> 04:06.000 I used the Pearl tests, the standard protests, 04:06.000 --> 04:12.000 as a target for test-driven development. 04:12.000 --> 04:18.000 And yeah, once the main objective was in place, 04:18.000 --> 04:21.000 having this basic language in place, 04:21.000 --> 04:25.000 I started going test by test, simplest ones first, 04:25.000 --> 04:32.000 and then the most complex, until we get enough Pearl working. 04:32.000 --> 04:36.000 This is now one and a half year on. 04:36.000 --> 04:41.000 And yeah, it started with a discussion with architecture on 04:41.000 --> 04:44.000 chatGPT and how to build the documentation. 04:44.000 --> 04:47.000 So there's a lot of things going on. 04:47.000 --> 04:49.000 It's not an AI project. 04:49.000 --> 04:55.000 I made this myself, but I asked AI for help. 04:55.000 --> 04:58.000 For example, very frequently, 04:58.000 --> 05:02.000 feature was implemented, the broken other feature. 05:02.000 --> 05:07.000 And AI helped figure out, okay, this is what this is actually 05:07.000 --> 05:09.000 in this to be done. 05:09.000 --> 05:13.000 So what is the logic to make those features compatible 05:13.000 --> 05:18.000 in implemented in the same source code? 05:18.000 --> 05:23.000 And you see, I didn't mention looking at Pearl source code, 05:23.000 --> 05:25.000 and never looked at Pearl source code. 05:25.000 --> 05:27.000 This is all made in terms of behavior, 05:27.000 --> 05:29.000 in terms of test-driven development. 05:29.000 --> 05:35.000 So what the tests show me, I need to implement. 05:35.000 --> 05:40.000 So currently, we have a number of CEPAN modules. 05:40.000 --> 05:45.000 CEPAN is a repository for Pearl contributions. 05:45.000 --> 05:50.000 And currently, Pearl and Java runs DBI, 05:50.000 --> 05:53.000 which is database connectors. 05:53.000 --> 05:56.000 It has HTTP support. 05:56.000 --> 06:01.000 It has several file types support. 06:01.000 --> 06:07.000 And many CEPAN modules, Pearl, Pearl modules, 06:07.000 --> 06:12.000 also run directly in Pearl and Java. 06:12.000 --> 06:18.000 The standard Pearl has interface with the C. 06:18.000 --> 06:20.000 It's called access. 06:20.000 --> 06:24.000 And Pearl and Java also has access, but it connects to Java. 06:24.000 --> 06:28.000 So, for example, for DBI, 06:28.000 --> 06:32.000 it connects to JDBC directly. 06:32.000 --> 06:40.000 It doesn't need the internally conventional Pearl uses database drivers, 06:40.000 --> 06:45.000 but in case of Pearl and Java, 06:45.000 --> 06:47.000 it only needs one driver, JDBC. 06:47.000 --> 06:50.000 So it was really easy to port. 06:50.000 --> 06:54.000 It's a large, relatively large file on Java, 06:54.000 --> 06:58.000 but it's a single database driver. 06:59.000 --> 07:03.000 So let's go to features. 07:03.000 --> 07:09.000 I built this feature matrix for myself to keep track of what's going on, 07:09.000 --> 07:12.000 how much of Pearl is implemented. 07:12.000 --> 07:16.000 And currently, it has a lot of greens, 07:16.000 --> 07:20.000 and a few reds. 07:21.000 --> 07:28.000 Red is things that are not really compatible between Pearl and JVM. 07:28.000 --> 07:32.000 For example, this dry blocks, 07:32.000 --> 07:37.000 this dry is called when a variable is no longer used. 07:37.000 --> 07:41.000 It's when Java does garbage collection, 07:41.000 --> 07:47.000 but there's no hook in Java to tell me when a variable is destroyed. 07:47.000 --> 07:49.000 Currently, no dry blocks. 07:49.000 --> 07:51.000 There are worker rounds, 07:51.000 --> 07:54.000 but I don't want to go there. 07:54.000 --> 07:56.000 At least not yet, or maybe optionally, 07:56.000 --> 08:00.000 because it would make the language slower. 08:00.000 --> 08:05.000 Yeah, and access itself is different. 08:05.000 --> 08:08.000 It's Java related, not C. 08:08.000 --> 08:11.000 So it's a simple module that you see. 08:11.000 --> 08:14.000 We'll not be directly vulnerable. 08:14.000 --> 08:16.000 But, yeah, AI can fix that. 08:16.000 --> 08:21.000 I already pointed a few modules, and it's relatively easy. 08:21.000 --> 08:25.000 C integration is actually doable. 08:25.000 --> 08:31.000 There are hooks in Java to call C modules, 08:31.000 --> 08:36.000 so that's not really a big problem. 08:36.000 --> 08:39.000 Yeah, things partially supported. 08:39.000 --> 08:44.000 I think most important is rejects. 08:45.000 --> 08:49.000 So there is a preprocessor in Pearl and Java 08:49.000 --> 08:55.000 that transforms the Pearl specific rejects 08:55.000 --> 09:00.000 into to be runable by Java rejects engine. 09:00.000 --> 09:04.000 But there are some things that Java rejects engine, 09:04.000 --> 09:05.000 just cannot do. 09:05.000 --> 09:08.000 So I think maybe in the future, 09:08.000 --> 09:10.000 the Reimplementer rejects engine, 09:10.000 --> 09:15.000 or use something like rubby, library, 09:15.000 --> 09:18.000 it'll come in some way. 09:18.000 --> 09:22.000 But a lot of rejects are already in place. 09:22.000 --> 09:25.000 Yeah, some things like back tracking controls, 09:25.000 --> 09:27.000 they don't exist in Java. 09:27.000 --> 09:29.000 And also some optimizations. 09:29.000 --> 09:32.000 There are some special cases in rejects that they take a long time. 09:32.000 --> 09:36.000 In Java and in Pearl it's fast. 09:36.000 --> 09:40.000 So maybe custom rejects engine actually makes sense. 09:40.000 --> 09:44.000 Yeah, for operators and statements, 09:44.000 --> 09:48.000 this is all done, all implemented. 09:48.000 --> 09:55.000 Yeah, smart match was deprecated, so not there. 09:55.000 --> 09:57.000 Let's see what else. 09:57.000 --> 10:05.000 Yeah, special control flow operators, Pearl has go to. 10:05.000 --> 10:10.000 And it can do some really interesting stuff. 10:10.000 --> 10:13.000 Some things are actually implemented, 10:13.000 --> 10:15.000 but not everything. 10:15.000 --> 10:20.000 And yeah, and then party modules. 10:20.000 --> 10:26.000 There is, yeah, IO layers is something Pearl supports 10:26.000 --> 10:29.000 to read files in different encodings. 10:29.000 --> 10:33.000 It's supported. 10:33.000 --> 10:35.000 Variables modules. 10:35.000 --> 10:39.000 Yeah, like I said, DBE is there, DBI. 10:39.000 --> 10:43.000 Pragmas version features. 10:43.000 --> 10:45.000 Features is something in Pearl that allows it 10:45.000 --> 10:47.000 to be backwards compatible. 10:47.000 --> 10:51.000 So you have a old Pearl program. 10:51.000 --> 10:57.000 It features and what experimental features. 10:57.000 --> 11:01.000 You can make it make Pearl behave as a specific version. 11:01.000 --> 11:06.000 And finally, I think most important at the moment is 11:06.000 --> 11:10.000 Java segment size limitation. 11:10.000 --> 11:18.000 Java originally was implemented as system B64K blocks. 11:18.000 --> 11:26.000 And if you try to compile a method that gives more than 60,000 bytes, 11:26.000 --> 11:29.000 it dies with a method too large. 11:29.000 --> 11:32.000 But Pearl doesn't have this limitation. 11:32.000 --> 11:37.000 So the fix is to split methods into smaller methods. 11:37.000 --> 11:41.000 This is already implemented, but has still some problems. 11:41.000 --> 11:45.000 And finally optimizations. 11:45.000 --> 11:53.000 Yeah, I'll make a small DBI test. 11:53.000 --> 12:00.000 Yeah, let's open this example, DBI. 12:00.000 --> 12:05.000 Yeah, so this is standard Pearl. 12:05.000 --> 12:10.000 The only special thing here is the database connection string 12:10.000 --> 12:15.000 is start with JDBC. 12:15.000 --> 12:19.000 Yeah, and everything else is exactly the same as Pearl. 12:19.000 --> 12:22.000 This actually runs in Pearl or Pearl on Java. 12:22.000 --> 12:26.000 If you have the right driver. 12:26.000 --> 12:32.000 Yeah, I think it's interesting to show as a last thing. 12:32.000 --> 12:42.000 Because it has enthusiastic people doing testing the limits of the syntax, 12:42.000 --> 12:45.000 Pearl can do things like this. 12:45.000 --> 12:48.000 This is toy in Pearl. 12:49.000 --> 12:53.000 You write a program that is just for fun. 12:53.000 --> 12:57.000 And try to make it as different as possible. 12:57.000 --> 13:03.000 And the game is that this should print another Pearl hacker. 13:03.000 --> 13:08.000 And with J Pearl can actually understand this. 13:08.000 --> 13:14.000 So it shows how the syntax integration actually works. 13:15.000 --> 13:21.000 Yeah, the success story is performance is actually very okay. 13:21.000 --> 13:24.000 It's about the same as Pearl. 13:24.000 --> 13:31.000 And it's now running a quarter million unit tests from Pearl. 13:31.000 --> 13:36.000 If you run Pearl with all the methods, all the modules, 13:36.000 --> 13:38.000 it's about two million tests. 13:38.000 --> 13:41.000 But the core Pearl is not that big. 13:41.000 --> 13:49.000 So it's over, I think, 95% of Pearl Core is already in place. 13:49.000 --> 13:52.000 We probably don't have time for questions. 13:52.000 --> 13:54.000 Maybe one question, no questions. 13:54.000 --> 13:56.000 Then we can talk later. 13:56.000 --> 13:59.000 And yeah, thank you.