James Simone, who joins to talk about DML Mocking, is the lead Software Engineer at James Simone LLC.
He is also a climber, runner, sourdough bread baker, oats enthusiast, and software developer. He’s the self-published author of the sci-fi / fantasy novel A Good Thing Never Stops.
Sponsor
This episode is sponsored by Copado.
Copado is the first Salesforce Delivery Management solution and the #1 native DevOps platform for Salesforce.
Copado DevOps 360 is the first Salesforce Delivery Management application, enabling you to measure the performance of your Salesforce implementation.
Main Points
- James’ the Joys of Apex blog series
- What is DML mocking?
- Is DML mocking important to implement?
- How to switch to using DML mocking?
- Is test speed also important for development?
- How DML mocking is created in a high-level picture?
- What challenges have James encountered when implementing the DML mocking lib?
- Does the DML Mocking lib support UoW?
- James’ SOQL and SOSL mocking implementation
- James’ thinking about code
Links
- James’ LinkedIn
- James’ Twitter
- James’ the Joys of Apex blog series
- DML mocking repository
- James’ article about the DML chunking error
Video Teaser
Contact Me
If you enjoy this podcast, please reach me out to let me know your thoughts or simply say Hello. My (Xi’s) contact info:
Take care, until next time :)!
Transcript
AI-generated, unedited.
[00:00] This episode sponsored by Capado. Capado is the first Salesforce delivery management solution and the number one native DevOps platform for Salesforce. Capado makes the jobs of Salesforce admins and developers fun and easy, even in the largest, most complex works. Plan and collaborate on work, then track your changes right on the user story with an admin-friendly Git interface. Automate development and testing, and track metrics so that you can target improvements. Capato DevOps Three Sixty brings advanced analytics such as value stream mapping, executive summaries, and rich interactive dashboards. These insights help you find inefficiencies and delight your users by delivering innovation faster than ever. Take the pain out of the Salesforce development process and make Salesforce development fun again with Capado.
[01:11] Hey everybody! This is Xi Xiao. This is yet another new episode of Salesforce Way podcast. Today I'm sitting with James Simone. Hello, James. Hey Xi, how's it going? It's pretty good. Welcome to the show. Thanks. Thanks so much for having me on here. So James, would you like to introduce yourself? Yeah, I'm James Simone. I'm newly relocated citizen during the COVID nineteen lockdown to the great state of Colorado in the United States. I have been working on Salesforce-related projects for going on eight years now. I run my own consulting company at the moment, focusing on bringing companies into Salesforce. And yeah, I'm super excited to be on the show today. I've listened to a couple of the Salesforce Way episodes, and I'm just blown away by the the talent that you've managed to interview over the course of your your career doing this. It's great. Yeah, thanks. Thanks for that. You know, I get your information from one of my listeners, Fabri Taringa. So I just want to give a big shout out to him. He linked me to your blog. You write a series of a blog posts. It's called the Joys of Apex. So I had the chance to really read the articles there. I was really blown away with the information you provide there. So I really, really
[02:41] enjoyed reading those article content. It bring me back the joy of reading, like Dan Appelman's book or Andy Fawcett's book. So I really learned a lot from that. Just to let you know. Well, thank you. That really means a lot to me. And you know, I I do consider myself to be a writer. I think it's important to focus on education, especially in the Salesforce ecosystem, because you know those those legends that you just mentioned they've sort of gone before us. They're they're a little bit less productive in the Salesforce ecosystem in terms of producing content these days. But there are people learning. How to use Salesforce and how to use Apex in particular. Every day, every day there's new people learning, and there's less in the way of relevant and new information on how to do so outside of Trailhead. So I've really been trying with the joys of Apex to bring people something that's new and fresh, and looking at something that I think a lot of people have sort of ridden off. For various reasons, but I love Apex for all of its limitations, and I think that those limitations help you actually to focus on how to accomplish things. So. Yeah, yeah, it's really amazing. You did this, you know. You you spend so much time writing those articles and share the knowledge. I really appreciate that. One thing is that I know you created a so-called DML
[04:11] mocking library, and you spent a couple of articles really in the Joy of Apex series to talk about what it is and why you created such a thing. So I want to bring all the listeners onto the same page. So would you mind to introduce what is this DML mocking at the start point? Yeah, absolutely. So. DML as as it pertains to Salesforce is the insertion, creation, updating, upserting, deletion, undeletion of records, and DML mocking is. The act of removing the actual Salesforce database from your Salesforce unit tests. So it's a a concept of separating out the database from your tests to help speed them up. Hmm. So I understand there is this HTTP mock. So which means when we write a testing Apex, we are not allowed to call out with this HTTP. So that's one of the reasons we have to use the HTTP mock to somehow create a dummy returning data. So this DML marking is kind of a similar concept. So when we in the test when we need to insert or update certain records, including delete and delete as well, we don't really interact with the database. Am I right? So it's kind of like a a mock, as
[05:41] as the name says. Yeah, that is correct. Okay, but why did you spend time on creating such a DML mocking library? Is it important? It depends. It depends on your use case. It depends on how many tests you have. I was working for a fairly large company at the time. And the issue that we were dealing with was specific to deploy time, which was starting to impact the developer experience, particularly because there were a number of different settings between the environments, between the sandboxes and the production level instance. And what we were finding, actually, as part of the deploys, is that. A deploy to production could fail for reasons that were separate from the unit tests and more. More specific to data that was in only the production instance, something that became a huge pain point was if you were working on a feature and you went to deploy it, or even worse, if you were working on a hot fix or something that really needed to be deployed asap, and you went to deploy it, and the deploy failed after running for an hour or two hours, you had lost that time, you know. It was something that you're busy monitoring all the time. Like, did this deploy succeed? It's something that takes you away from
[07:12] what you're doing, constantly refocusing your attention on this. I think everybody's familiar with the Deploy tab in Salesforce, and you're watching these tests run, and you know you see a test fail. And you know that you have to start over. It's a pretty painful procedure. Yeah, I heard the similar kind of explanation from Kevin Jones, who is working in Financial Force. He also told me that the Apex testing take away too much time now, which could be mocked up and speed up the time consuming. So I think he shared the same opinion as you do now. Yeah, I don't think it's a a very revolutionary thought. You know, I'm certainly not trying to make the claim that I'm the first person who's thought of these things. I'm certainly not. I feel like it's something that I had a specific use case for improving on, and I actually was lucky to have the time to work on improving this because a lot of the times I think we have these pain points as developers, but we don't get the chance, or we don't make the time. To make those improvements, I I see not only for the test. I mean, each time when we need to do the DML and interact with the database, it it did take a lot of time in the production code as well. Right. So of course, when we mock it out and we don't need to communicate with the database, the time drastically
[08:42] reduced. Do you have some data, like a statistic, telling when we run Apex tests with the DML or without the DML using the DML mocking? What's the time difference? Yeah. So it's a great question. I'm I'm glad you asked. It's something that. Obviously, I think that the biggest input into how long DML takes when you actually use the database is the number of rows that you are acting on. I like to use the the synchronous Salesforce limit of ten thousand DML rows when doing. Tests in order to validate performance. I think to some extent that is an artificial benchmark, as all benchmarks ultimately are, because it's unusual. I think probably for people to be actually. Manipulating that many rows at any given time, especially knowing that if you, you know, have one extra row in your dataset, now you've you've encountered a Salesforce limit exception. But in looking at ten thousand rows, interacting with the actual database as compared to DML mocking, I've seen. A ninety percent improvement in testing times versus using the actual database. So, if it takes ten seconds to perform an insert on ten thousand rows, oftentimes it takes less than a second to do the same operation with DML
[10:12] mocking. Hmm, that's really like a drastically improvement on the test speed. So it totally makes sense to me that now you you need to create this DMM marking. So let's also ask because I see a lot of testing, piloting, open source project. I know your your project is also open sourced in the GitHub. So I will put the link in our show notes for our listeners to check. But Is that a lot of customers of yours are using these mocking libraries as well, or is it just a testing side project? There are there are several customers who are using it. Yes, I think what I've tried to emphasize in the joys of Apex is that it's not necessary or even advisable if you are working on something that isn't a greenfield project to do a huge overhaul, a one-time adoption of this method. These are. These are theories and approaches to testing that can be incrementally implemented, and you can start seeing results in your next pull request, in your next feature that you're planning. These are thought processes that help you to improve your testing time, but it doesn't have to be you know this huge overhaul. You have a pull request that has you know 700 changed files, and I think that. That kind of drastic
[11:43] change. If you're really concerned about testing time and it's something that you really want to focus on, sure, you could do that, and you would immediately see a massive increase or decrease in your testing time. But I think that it's more expounding on the theory and helping people to realize there are things that you can do that will help speed up your the next test that you write. That's good, because I kind of hesitate when I need to switch to a big, heavy library, for example, to FF Lib. Which means I totally need to change a lot of things in my existing codebase. But for this DML marking, we can gradually adapt into this marking if we want. That's what you want to say? Exactly. And I had the same concern with FF Lib, which is one of the reasons why I didn't end up using it. Okay. I get the idea, especially for the deployment part. The Apex test can take really long time. For example, I know that for several hundred Apex testing, it can take even up to an hour. Which means when I deploy something to production, I'm just waiting there for an hour or taking coffee, taking break. That I totally get. I think a lot of developers are having the same experience. But what about for the development? So let's say I'm writing several Apex tests and I need to send back to Salesforce Cloud and then run the test, which doesn't really consume that much time, does
[13:13] it? It depends. If you're running every single test that you have in order to test one specific change that you've made, sure it could take a long time, right? I think probably it's more the case for people that they tend to only run the tests that are. Directly touching the feature that they're working on, right? Especially when you know, are my tests going to take a long time to run? I'm not going to run the full test suite just to make a little change. I am going to run the tests that are immediately related to this, but you know, I can't tell you how many times. Over the past few years, I I've worked on a feature and run the tests that I thought were directly related to it, only to find a failure elsewhere at the the end of working on it. Right, so it's like even when you have a really tight code base that you feel like you understand and you are working on something that's very specific, it's always surprising to see the things that end up going wrong. Hmm. Yeah, for sure. If the whole test suit just to take, let's say, less than ten seconds, then why not, right? I want to implement a new feature or refactor the code. I can just run it within ten seconds, and it doesn't consume too much time. And I'm sure then it's guaranteed that I didn't break something somewhere else. Exactly, and I think that ideally speaking, we would all be able to do that,
[14:43] right? I see. So. That's quite interesting with this DMM marking and the library. So, would you mind to quickly introduce us how you did it? Absolutely, and I think the the answer is much more simple than you even might expect. It's writing a very small wrapper around the actual database manipulation and replacing the calls to do inserts or updates, what have you, with corresponding lists of S objects. So those those calls to do the database manipulation actually just end up in a list that's kept in memory. That's the that's the entirety of it when you summarize it. So you mean you create a wrapper which is like a actual layer between your code and the database, so sitting in between. And whenever the production code that does need a real database insertion, then you call the real DML. Let's say the insert or update records. But in case it's in the test, there's an easy way for us to swap. This a database in action, and then using your mock. That's what you did. That's correct, and I would also you know stress that the.
[16:13] The way of performing that switch and test is something that's only possible to do while testing. We're very lucky, as part of the Salesforce framework, to have this test visible annotation. On objects, and I think that actually it makes it really nice for the developer when you are focusing on creating this single seam by which people can perform the hot swap between the actual DML and the the fake DML that's happening in test, because you know 100 for sure that it can never be called in production level code. Ah, so you use this annotation is test visible. Because that can be only used in the test code, so it's a safety net for us, so we don't accidentally bring it to the production code. That's right, because that would be a real shame if you thought that you were performing updates and it turned out it was only happening in memory. That's a good thing indeed, because we in the team we have junior developers. If they didn't really fully understand how to use it, they might do something like accidentally, right? So, okay, that that's always a good thing. I also heard, like, even read the book from Dan Appelman's Advanced Apex book. He he introduced created wrapper for the custom settings, created wrapper for the custom metadata types. I would assume this is kind of similar design pattern. Absolutely, and you know I use
[17:44] the the wrapper for custom settings that Dan talked about in that book on every project I've ever worked on. Okay, I think that understanding the the purpose of wrappers and using them correctly is one of the the fundamentals in being a good developer. Hmm. Okay, and I understand in this market library, you bringing also this factory pattern in the design part. Am I right? That's right. Why do we need the factory pattern? Because because the reason is that I know a lot of people don't like it because it sounds really heavy. You can easily instantiate a new classes, and instead now you have to always to say, okay, I have the factory, always ask the factory to get a new instance. Yeah, I know it's like it's interesting, right? Like I feel as though I have this crusade against the keyword "new." I'm trying to discourage people from using the the new keyword when writing production level and test code in Apex and. For me, I know that the reason for that is I wanted to have again this single seam where I only was going to use one approach to perform this DML mocking in tests, and there are other. Things that I get into in the Joys of Apex about replacing SOQL and SOQL calls
[19:14] as well, which is one of the other big forms of slowdown when it comes to running Apex unit tests. And having that factory singleton that is solely responsible for the creation of objects helps with creating a consistent method of performing that that quote unquote hot swap between the mocks and production level code. I I want to be very clear. I don't think it's the only approach that works. It's the approach that I happen to like. I know that people don't like the factory pattern, and I can totally understand that. So I think that there are a lot of different ways that you could do this, but the consistency of it is what really appeals to me, and it's helped me to write unit tests that. Always perform that hot swap in a consistent way. Okay, I see. When you create this DML mocking library, you just mentioned that it's just a wrapper, and it sounds really simple. But did you encounter any challenges when you implement this library? I like to think the limitations that you experience in working with Salesforce help to focus you on how to solve the problems that you have. But yes, absolutely. To answer your question, the the bummers that I experienced
[20:44] in creating the library, there's a couple of them. I think the biggest one was that. Using dependency injection correctly, in order to create dependencies for objects that were going to be able to be used consistently as either a mock or in production level code, I found that the factory pattern was the best way to do that. And I think that that is in the end a bummer because. You still have to figure out how you're going to perform dependency injection in order to best use this code, or best avoid the problem of always having to construct these objects with either a mock. Or the real DML wrapper in your production level code. So that's definitely for sure a big bummer in the sense that we don't have an existing dependency injection framework or even. You know a collection of frameworks for performing that dependency injection in Apex. When you look at a lot of the other object-oriented programming languages out there, there are a whole suite of dependency injection libraries that you can use to bootstrap your application, so that when it starts up. Your dependencies are installed and use a specific instance of of themselves
[22:15] on startup, and that's something that you just can't do in Apex. You have to know the type of the object beforehand. In order to to do any kind of testing or production level code, so that was a a bummer for sure. I think something else that we've talked a little bit about in email was the chunking issue. That's something that handling the chunking issue, which just to back up a step is you're not allowed to perform DML on S objects if they Exceed a chunk size of ten, and a chunk, the chunk, is incremented every time an S object type changes between objects in the list. So, if you have an account that you are updating, and you are also updating an opportunity, now your chunk size has increased to two. And if you are alternating accounts and opportunities, you can only perform. That alternation five times, right? As you were you were pointing out to me, it's actually not the case that you're going to run into ten. Objects before you encounter this error, it's going to be five objects if you're performing an alternation every time. So if you know that you're updating this list of accounts and opportunities, you can only do so in a very specific way without sorting the list and handling that
[23:45] problem was something that that was a bummer for sure. Hmm. Just to let everybody know, the context here is that because we're creating the DML mocking library, for example, we need to mock a insert action, right? A lot of time we need to insert, for example, a hundred S objects, and in this library we do have a function, a public function called do insert, which is the mock of the the insert, and the do insert has the input of a list of S objects. And because the input is a list of S object, you can give in couple of accounts, couple of contact, couple of opportunities. You can all feed them into this list. But when we really do a Database insert for this list of S objects. There is this hidden rule that you cannot auto late too many times. That's the limitation that you hit. Yep. Thank you for clarifying that. I don't even know if I clarified because it's really hard to yeah to make sure that yeah. But we will put the link to our show notes. So the chunking size actually is the first time I I heard about it. I I never knew it before. Well, I want to be clear. It's something that I only learned about after experiencing that exception in production. You know, so it's not like I knew about that beforehand. It's something that I encountered after having already
[25:15] implemented the DML mocking library, and because all of the calls to the database that I was making was going through this wrapper. It was something that had to be handled, and to be clear, you would have to handle it, even if you were just using the Salesforce database manipulation language. If you were calling database dot insert or just insert, and you hadn't sorted your list beforehand, you would get that exception anyway. So this was an attempt to create a safe way of always performing inserts, but it's something that you have to handle as a developer. In any case, if you're in this experience of having a varied list of S objects that you're performing DML on. Because most of the time, as a junior developer like I am, I just do insert accounts or insert contacts. I won't really insert as object. Yep, and I mean it makes sense, right? In in In the beginning, you want to make use of those, those features like just calling insert on a singular list or on a singular account or what have you. You start to think more about performing inserts or updates on a varied list of S objects when you start to realize the performance implications of performing something like that in three different transactions or two separate transactions. That's actually one of the reasons why people really love using the list of S objects to do
[26:46] all of the DML in one specific call because it it increases the Efficiency of your code. So, since you are talking about this topic, it reminded me this unit of work that was kind of get populated by Andy Fossett in the FFLP. They have the unit of work implementation. So, does your library handle this kind of thing? No, it does not, and it's it is funny that you ask though, because a good friend of mine and somebody who I worked through a lot of the issues with this DML mocking library with, Jonathan Gillespie. He has a unit of work framework that he has built to handle that sort of operation. It's a little too much, I think, for me. I'm looking to do something a little bit simpler and a little bit less heavy in terms of implementation. I I can definitely understand why people want to do things in that way, but. It was a little bit outside of the use case that I had. I see, I see. Just for the listeners who are not familiar with unit work, I have a episode in the past with John Daniel. Check that episode. You also mentioned you have touched that SOQL as well, right? You can even mock up the SOQL
[28:16] queries. Yep, that's correct. And again, that's through the use of another wrapper object. I talk about that a little bit less than I talk about DML, because I think that the performance implications are much more severe for doing DML versus doing the calls to SOQL. I happen to like it because it allows me and my tests to validate the. Output of a query, which you would otherwise not really be able to do. So, if you want to validate, for example, that you are filtering for something specific by using a where clause or a group of where and statements in your SOQL, you don't have a way of of doing that unless you are using a string to perform your SOQL. But that that sounds, you know, a little bit. Dangerous already. As somebody who's working in a strongly typed language, you should be looking at strings with some suspicion already. So. Doing SOQL mocking for me is a little bit more about validating that the query that's being performed is correct than it is about the performance implications of doing the mock. I do like the fact that I I don't have to do inserts. I don't have to do these these huge chains of related objects in order to
[29:46] get. At the results of something when I'm querying for them in my unit test, but I think that that that's sort of a ancillary benefit. That the big thing that you get when you do SOQL mocking is just being able to validate that the query is correct, which is big. Yeah, I see the point. Is that only you created the DML mocking library, or do we already have those others in the ecosystem? The one that I wrote for the joys of Apex is entirely my own. It is certainly based heavily on other work that I've seen. Again, my my friend Jonathan Gillespie is somebody who created. Actually, it's it's interesting because he used this phrase Nebula Framework, which is different from the Nebula Core that you were discussing with Aiden Harding. So it's definitely heavily influenced by his work there. Again, it's something that for me, I wanted something that was a little bit more lightweight, and I went. To great lengths to write something that could be used much more easily with a lot less in the way of classes and interfaces that needed to be implemented. Hmm. I see. So I spent some time to read your code in the GitHub. One thing I paid attention is that you have a really good skill on object oriented thinking. So you know when exactly to use, for example, interface. When
[31:16] to use the virtual classes. When to use the static method. You know all those things around object oriented. In the end, when I see the full picture of the solution, it's a really nicely encapsulated art of work. I would say. So, how did you gain such an experience? Because most of the developers, they don't have that much of strong experience or mindsets against the object-oriented in Salesforce ecosystems. Yeah, it's a good question. I think I've spent a lot of time thinking about code. I was reading about code before I wrote code. I I read Clean Code in I don't know 2009 or 2010 before I had written a single line of Java. I read Clean Code and I didn't understand it at all, you know. But I went back to that book in particular a couple of years later, and it was very interesting to. Come back to it and realize that I had remembered parts of it without realizing it. That I had sort of absorbed the information in a really organic way. I don't want to use the word osmosis, but you know, there was something that was absorbed that I hadn't realized that happened subconsciously. And that book in particular, I really recommend to developers who are starting their their programming
[32:47] journey. There.
[32:51] There's a lot of books out there. I think that the way that people learn is so different, so it's hard to say like this worked for me, and I think it's going to work for you. I I don't think that actually. I think that everybody learns differently and and comes to appreciate code differently. Yeah, the clean code book is from Martin Uncle Bob. Martin Uncle Bob. That's right. Yeah, I always remember Uncle Bob because in the episode seven, I had a chat with Robert Sussman, who is also a big advocate for the clean code concept and the book. So, if our listeners don't know what it is, you can also go back to the episode seven or eight to listen to Robert Sussman's sayings. All right, great James. It's a nice chatting with you. I really enjoyed again. You know the joys of Apex. I hope you will continue writing it, or maybe even one day you can, you know, summarize everything and even create a book out of that. So I would really love to read your book. Thank you. That means a lot to me. I hope to continue writing it. It's been really a joy so far, so to speak. And yeah, I really appreciate it. Good. Excellent, James. So talk to you next time. Bye bye.
