Aidan Harding joins to talk about Nebula Core, a Salesforce Apex functional programming Library, an open-source project published by Nebula Consulting company.
Aidan is a senior Salesforce consultant and developer.
Sponsor
Proton Text is a complete SMS texting solution built for the Lightning Platform, and by one of our previous guests Pat McClellan.
Main Points
- Aidan’s self-introduction and the Nebula core open-source library including Salesforce functional programming feature
- Feature 1 – Cached way to retrieve record type information
- Feature 2 – the functional programming part in the lazy iterator
- Feature 3 – the lazy part in the lazy iterator
- Feature 4 – Sobject index
- Feature 5 – Custom metadata type-driven logger
Links
- Aidan’s Twitter
- Aidan’s LinkedIn
- Nebula Core repository
- Blog thread about Feature 2 – Apex functional programming
- Lazy iterator sample code
- Video – Aidan recommended Lazy evaluation talk
Video Teaser on Nebula Core Library
Outro Music
Shadow of Ninja – FC/NES Game.
Contact Me
If you enjoy this podcast, please reach me out to let me know your thoughts or simply say Hello. My (Xi’s) contact info:
Take care, until next time :)!
Transcript
AI-generated, unedited.
[00:00] Our sponsor today is Proton Text, a complete SMS texting solution built for the Lightning platform by one of our previous guests, Pat McLellum. Here is reason number one that admins and users love this app. It's lightning smart. Admins can easily configure the one-to-one messenger component for private messaging, or collaborative team texting using queues or shared phone numbers. And using hasRecordId, the component always knows where it is, which conversations you want to see, and which people you might want to text. Proton Text uses Lightning Platform features and Apex logic to make the app intuitive and easy to use. It's not magic, just good design. Learn more at protontext.com.
[01:08] Hey, everybody! This is Xi Xiao. This is yet another episode of Salesforce Way podcast. Today I'm sitting with another new guest with me. His name is Aidan Harding. Hello, Aidan. Hi, Jay. Thanks for having me. Sure. So I invited you today mainly to talk about this open source library your company had. I think you also contributed a lot, so you're the main contributor. Yeah, I. It's something that I written for myself. Pretty much all of it, and then we work together on ideas on how to develop it. Yeah, and hopefully, as we. Move forward with it. Some of the other guys will be able to contribute bits of code to it as well. Yeah, I had a look at it. It has really special features which we haven't seen before somewhere else. So that's why I want to get you on the talk and then you know introduce to our listeners. But before we jump into our main topic, would you still want to introduce yourself? Yeah. So as you say, I'm Aidan Harding, and I work for Nebula Consulting. We're a consulting partner with Salesforce, mostly based in London, and we do kind of specialise in ParDot as well as Salesforce. But myself, I mostly work on the Salesforce side. Okay, all right.
[02:38] How should we start talking about this open library? Should we start with why, or should we start with the what? I think equally they're important. Yeah, yeah, it's probably it's hard to say. The two things sort of run into each other. So let's talk about what. So what this open source library contains.
[03:01] There, there are kind of there are a few different types of thing in there. So sometimes it's things where there's a piece of functionality that feel like Salesforce probably should have done, but isn't part of standard Apex, and you just want to add it to bridge that gap. Okay. And sometimes it's very small things. Like one of the things in the library is something that is a sort of cast way of retrieving record types. So I can't even remember how long ago it was, but you used to have to do a SOQL query to get the record types associated with a particular type by name, and then you could carry on and use them. And so you typically want some way of caching that of. You load up that record type, and then any other piece of code that needs to use it can just get it from that cache rather than having to query it for it by itself. Okay, so you create a a cache layer, right, by yourself, so that instead of go back to the server, you always use the cache layer if the data remains. Yeah, I mean it's all in Apex, so it's all running on the server. But it's just used to write in a static variable, and everyone has access to it. You only query it once, and it kind of it gave us a consistent way of getting hold of record types. And then, part of what's nice about having a library is eventually Salesforce added
[04:32] to the system library, so that you didn't have to do a SOQL query to get those record types anymore. You could just get it from the schema, and so we change it once in the library, and suddenly. All of our code that has the newest version of the library has saved itself one query on your limits, which is great. Okay, I think there are other libraries, open source libraries, are doing similar things like a caching. I think it's up to the listeners to really check the code how you did it. I think that's more important. When you use the tool, you better understand how it made it. You may not need to rewrite the same functionality, but you want to understand how it worked, right? Yeah. But I think one of the most important things or most interesting point for me is the lazy iterator. Yeah, you know what it is, right? Yes. Yeah. Let's talk about the lazy iterator. Yes. Okay. So this is um, a collection of iterator operations in Apex. So it's kind of inspired with. When Salesforce got us all writing Lightning components, we could all work with JavaScript, and then you can see JavaScript has these nice array operations of filter and map and reduce that allow you to deal with arrays instead of writing a for loop. To process the
[06:02] array, you can call these functions. So filter picks out which elements from the array you're interested in. Map maps individual values, so you can produce a new array with new versions of the values. Reduce is a kind of accumulator function, and when you write code with those sorts of operations, it's at a higher level of abstraction than if you just write the for loops. So if you're familiar with that kind of idiom of using filter, map, and reduce, you can more easily see what code written in that way is doing. Whereas in a for loop, you kind of you have to break down that. Construct of what's this for loop doing? What's that if doing within it? And it's harder to figure out what's going on. So I wanted to find a way of bringing those sorts of operations into Apex, and wanted to make sure that it was efficient. And I kind of wanted to make Apex act as if functions were first class objects in Salesforce, which is in modern Java you can pass around functions the same way as you can in JavaScript, which is really useful. And in in Apex you just can't, so I had to come up with a way of kind of faking that using interfaces. Okay, I didn't know how you made it. Let's step back a little bit because a lot of
[07:32] listeners they may not have the functional programming background, but the good thing is that most of us have already started touching the JavaScript. Right? You mentioned the JavaScript has those. Let's say collections. Let's say we have a collection of data. Based on the collection of data, you have the dot map. You have a dot filter, dot for each, dot slice. Those kind of functional way of manipulating the data. Yeah. If we are using Apex, the object-oriented way is that we always create a temporary, for example, map or temporary list or temporary set, right? And then we use the for loop to go through all the existing data and then save the calculated the data into that temporary map. So we do such things a lot in Apex. You see it everywhere. But in this functional ways that you don't really use the temporary collections or the follow, you work directly on the data set by itself. Yeah. So there are probably two things to think about related to that. So. In case you're not familiar with how these things work in JavaScript, an example of the kinds of things that you could do with the map function in JavaScript would be if you had a
[09:02] list of account objects and maybe you wanted to pull out one field from each of those objects and have a new list which is just the values from that field. In JavaScript, you would just take the array of accounts called dot map and pass in a function which just reads the field you're interested in. So we could get a list of from a list of accounts, we could get a list of account names. Just by calling the map function in one line, and if you're working in modern JavaScript, then you can use arrow function to define an inline function that's going to return the name for you there, and it's very slick and it's very simple. And that is the kind of thing that I wanted to be able to do in Apex because if you think of. The way that a lot of Apex triggers break down, they tend to happen in sort of three phases. Of in the first phase, you Go through all the trigger context lists, and you pick out some records that your trigger needs to act on, which is kind of analogous to the filter operation in JavaScript. And then, often you'll have to do some SOQL queries based on that to go and read some other data. And then, in a third phase, you'll construct some stuff
[10:32] that you're going to want to insert or update into the database. So you'll kind of from your list of trigger context records, you're going to want to turn those into new things that you're going to create and or update. And what I was looking for is an. The right level of abstraction to make that kind of standard shape of a trigger more obvious and more simple to write, and the lazy iterator part of the library was what I came up with to try and solve that. Hmm. So we want to reduce the code that does the flow controlling, like the for loop, and those intermediate variables, right? So those things are not directly related with the data you want to act on. So they're just the flow that you have to do. In the sense, it's more like the boilerplate code. And it just takes lines, and you have to read them through. So, but with this functional way that your open source library built is makes it more succinct, more precise. You you read the lines, you really know what you are doing. Yeah, exactly, and it's. I think it, in a sense, when we write the object or in it code, and we want it to be clean code, we use
[12:02] the private functions to wrap those details inside different functions. So we hide those for loop into sub functions and make sure the the main function really reads, and by the function names. So that's usually how in object oriented we write the clean code. But functional programming, we don't even need to do that. When we manipulate data, there's so many functions that act directly on top of the existing data. So it helps more to calculate the data. Yeah, and that was one of the things that sort of pushed us towards. This approach was really taking existing triggers and saying, "All right, we want to write this in a clean way, so we don't want any of our functions to be too long." And as you say, you give each phase of the trigger. A name that explains what it's supposed to do. But once you've done that a dozen times, you start to realise that the content of those private functions are often nearly the same. It's always a for loop within if, which checks whether or not a value has changed in the trigger context, and is the new value the one I'm interested in? And you're doing that thing all the time. Yeah, and so if you have some building blocks that allow you to express
[13:33] your intentions more directly, then you can write less code, and it's easier for people to to read. And you don't even need those private functions to split things up anymore because it's just so much more concise. Yeah, I think in the library open source there are a lot of examples. We will take some examples and put in our show notes. You know, once the listeners they could just open the examples and have a look at what the lazy iterator really means in your codes. So that's more easier for people to understand. Yeah. One thing I would like to explain about the library is where the lazy part of LazyIterator comes from. So,
[14:17] if you look at the JavaScript implementation of filter and map and so on, one of the things that was always a nagging doubt in my mind was is there an efficiency loss of writing your code with filter and map? As opposed to writing it just with a for loop, and it turns out that in JavaScript there actually is. So when you write filter on an array in JavaScript. It filters that entire array and returns an intermediary array. So if I take a single array and maybe I call filter on it ten times, and then I call map values, and that returns me my end result array, what JavaScript would do would be that it would generate ten intermediate arrays. And so you'd be iterating over all the items ten times. Okay, and that's obviously not terribly efficient. And so in JavaScript, you kind of have to be aware that you're trading off efficiency for clarity. And I didn't want to trade off that much efficiency for the sake of clarity in what I did in Apex. So. The fundamental unit of what my library is doing is not the array; it's the iterator.
[15:47] So, if you were to do the same thing and build up a sequence of ten filter operations and then a map. In my library, what it would be doing is it would be wrapping an iterator in ten layers, and each item would pass to all of those layers. So you're only iterating through the array once, but each item goes through all of those things. So you don't have the same bad performance hit as you have in JavaScript for that stuff. So the the lazy here the name really means that you don't really go through the items one function by one function, right? You somehow kind of combine them if there's no sequence, like a demand. So that's exactly the demand is the key thing, and that's where the laziness comes in. So you only pull something through this expression that you built up when you know you need it. So some of the functions in Lazy Iterator don't return a whole array. You might just say I want the first thing, and if you do that under the hood, what it does is it iterates through that array as much as it needs to until it can give you the first output value.
[17:13] Hey! Thanks for listening to the show, SalesforceWay Podcast. It's all the time looking for both guests. And topics. If you have guest recommendation or any topics you'd like to listen to, don't hesitate to let me know. My Twitter, LinkedIn, and email information is at the end of the show notes of each episode. You can also find it on the SalesforceWay.com website. I really, really, really need help from you on this matter because only you, my lovely listeners, know what yourself want. So, please help me invite great guests to talk on great topics, so that we spread great knowledge. Now let's get back to the show.
[18:09] This sounds really smart, you know the code. Yeah, so. How did you come up with this idea, and and what skills do you need to build this iterator things? The idea for doing it in Apex was just based on a talk that I saw about lazy evaluation in general. And so this particular talk was based in Java, so it was done in a modern version of Java with lambda expressions and things. And so who's explaining how all of that works in Java? And I thought, hey, I wonder if I can write that in Apex. And having having written the basics of it in Apex, it was was then. A series. Ah, it was quite a few months of thinking about it, developing bits of it to pull it towards something that I could practically use in production code. It was kind of nice to be able to replicate the examples from that talk that I saw in Apex. But what I really wanted to do is see if this could be really useful for us in in production. So I can I can put the link to that opportunity and the show notes because it's a really yeah definitely very good. Okay, I would I would definitely put that on my show notes, and I also want to take a look what it is. Yeah, um, in in the library, the readme it
[19:40] contains a lot of features. Let me just open it on my browser here. One of the things is that you mentioned you have a trigger framework. And it's driven or controlled by the custom metadata types, right? You can change the customer-related custom metadata types, and then change the functioning part of the trigger framework. I think a lot of trigger framework now in open source area are moving towards that. After we have in the make custom metadata types, and it's it's a nice way. It's a it's a neat tool to really control on the fly. That's a thing I think a lot of listeners have already know. Let's move forward. It's just a long list in your your library, you know. Some of them have the hyperlink to a specific documentation. Some of them just the text. For those text ones, are they already implemented, or are they to be implemented? Yeah, everything in that list is implemented, but if it doesn't have a link, it's not documented. Okay, so documents come later. Yeah, all right. Yeah, I kind of. I was quite excited by what I'd done with Lazy Iterator, and I just wanted to share it with people. I do that now, and then I'll get add more implementation. Yeah, that is definitely the number one things I like, and there are some other handy tools you created. For example,
[21:10] the S Object Index. Yes, so I understand a lot of time we need to create a map which is built with the ID and maybe a S object, right? The ID map to the S object, but in this S object index, what you are doing is that not only the ID field, you can use whatever field in the S L S object and create the map. Something like that. Yeah, that that's exactly right. So one of the another kind of common thing that we would see. In Apex is if we had a trigger on contact and we wanted to copy some details down from the account to the contact. What you would do is you would query the parent accounts, and then you'd put them into a map from ID to accounts. And then, as you loop through the contacts, you can pull those accounts out of the map, and you can put the values in. And that's what you were just talking about—the kind of standard thing that we're used to. But
[22:14] yeah, there are plenty of circumstances where. Id is not the only key you want to use. You you might want to use the industry of the account, for example. And as soon as you do that, you lose the uniqueness property, so it doesn't become a list, a map of industry to individual accounts anymore. It becomes a map of industry to list of accounts.
[22:41] And then, if you're going to try and build that up in your code, it means you have to iterate through all the accounts, and then you go to the map and you see if there's already a list in there, and if there is, you append it, and if there isn't, you create one with that account. And again, it's quite a lot of boilerplate code that you see in lots of Apex. Yeah. So I just wanted to find a way to. Create a very general solution for that, so we never have to manage our own map of anything to list of S objects anymore. So in this S object index, in the constructor, what you really need to do, you say this is the field name, right? And this is the list of object. Then create the map for me, and that just magically does it under the hood. Yeah, and the other thing that it does is it allows you to use multiple keys. So it doesn't just have to be a single field. It can be a list of fields that you use to retrieve stuff from the S object index. So this can be really useful for something like checking for duplicates. So if you imagine another kind of standard scenario, with
[24:03] maybe when a certain thing happens in your system, you need to create a case to follow up on it. And you want to make sure that you don't create a duplicate case if that has already been created, and. To tell whether or not a case like the one you're thinking of creating already exists, you probably have to look at a collection of fields. You probably, maybe, you have to look at the subjects. Maybe you have to look at the related contact and a couple of other things. So a pattern you'll often see in code that we have is we'll make a list of these cases that we're thinking of inserting. And then we'll do a query from the database to see which similar cases are in there. And because we, our criteria are on multiple fields, your SOQL query is going to return more results than just the ones that might be duplicates. And so you can put them on in the next object index. And then, if you use the one you're thinking of putting in, and say, "Is there already a case which looks like this?" Yes, object index will give you back a result and tell you whether or not there was something there or there wasn't. Okay, yeah, I got the point. So you have multiple fields indexed in this S S object index. And
[25:33] then, when you want to check the duplicates, then of course you go to those fields to to see if they are the same value in those fields, right? Then you detect whether it's duplicate or not. Okay. Yeah. That that sounds sounds good. Yeah. Some definitely. Sometimes we do need need this. Yeah, yeah. It's a class we use all the time. Another thing I'm just still checking the list, the long list here is that you have a logger class. Yes, I think a lot of libraries they have their own logger, which makes the life easier. So what's interesting here is that you also use the metadata type to control the logger. So what does it really do? Could you explain us? A general design principle that I'm keen on is postponing decisions as late as possible. And I kind of I wanted us to have a shared approach within the company to how we would call a logger, but I didn't really know what the right solution for logging was. So what I wanted to put into our core library was a single interface for logging, where you could plug in different logging implementations.
[26:52] And so that was part of the reason for using custom metadata. Is the custom metadata can specify an Apex class, which actually implements the logging. If that makes sense. So, if you think about the kind of ways you might want to log stuff, you might want to log it to System. Debug. Which is useful when you're writing unit tests and you're checking the logs. You might want to fire a platform event. You might want to write to a custom object, and it may be that the customer already has their own logging object, and you want to write to that rather than introducing your own. And that was one of the benefits of having this custom metadata-driven logger. We've got a single unified interface in Nebula Core, but if we need to write a custom solution for a particular customer, we can do that, or we can just have our own log object, which works for most people most of the time. Or if we have something where we need to write logs, even if the transaction doesn't finish, then maybe we can use platform events for that.
[28:04] And. Okay. The other really useful thing about custom metadata for that is it means you can change the logging granularity in production just by fiddling around with the custom metadata values. So it can help you to figure out those production issues. Okay, yeah, definitely. This is also one of the things I need to check afterwards. You know, you really need to see how it works and does it really fit into your own need. So I also recommend our listener to to go there to check. But the good thing always is the custom made data type allow us to configure according to the needs. So you don't need to rewrite the code or change the code. So you just change the settings on the record, and then it does as you need it. As long as we build the solution in a flexible way, yeah. One of the things that I've tried to do in writing the library to make sure that people can adopt it easily is to consider a lot of the implementation details in the library to be defaults rather than the absolute way of doing it. And sort of using custom metadata, which says we're going to run this class name, and that class name is usually something from Nebula Core, allows somebody who has different preferences or different needs to kind of swap something else in if that's what they need. Hmm, makes sense. Makes sense. It's
[29:34] easier for people to adapt if they don't want the default way. They can just change according to their own need. Yeah. All right. Thanks, Aidan. I think that's the end of our session. So really enjoyed talking with you, and I really enjoyed you share your thoughts with us. You know, these things I really like building the libraries that I think you definitely put a lot of thoughts behind this. And I also recommend our listeners to check your code because only by reading the code you understand how you are doing that. Like building the iterators, if I. I couldn't even think about it. I thought it's a functional programming part. You can easily build it in the object oriented, especially Apex is like old dialect of Java.
[30:23] Yeah, yeah, I would love it if people were to get in your contact, run the code, ask questions. Here's the room. Yeah, thanks a lot, Aidan. All right, thank you.
