35. The Power of Salesforce Asynchronous(1) | Jitendra Zaa

Jitendra Zaa shares a story of using Salesforce Asynchronous to fulfill the customer requirements.

Jitendra is Salesforce MVP, Evangelist, famous Salesforce blogger, Sr. Technical Architect at Bluewolf, 23x Salesforce Certificates.

Sponsor

Proton Text is a complete SMS texting solution built for the Lightning Platform, and by one of our previous guests Pat McClellan.

Main Points on Salesforce Asynchronous

The guest Jitendra shares a super exciting Salesforce implementation journey, where he and his team attempted to obtain as much as possible Salesforce platform potential by using asynchronous methods to fulfill customer requirements.

The story covers this and the following episodes.

Links

• Jitendra’s twitter
• Jitendra’s home page
• Jitendra’s blog
• Jitendra’s blog article relating to the story in this episode

Video Teaser

The YouTube Video URL

Transcript

AI-generated, unedited.

[00:00] Hi, our sponsor today is Proton Text, a complete SMS texting solution built for the Lightning platform by one of our previous guests, Pat McLehame. Here is reason number four that admins and users love this app: it's seamless. Whether you're blasting to ten thousand leads, or responding with a automated text, or having a one-to-one conversation with one person, the customer experience should be seamless. The Proton Blaster component lets you blast to thousands, and then configure automated responses and forwarding to queues and users for personalized follow-up. And all of this is as easy as creating a calendar appointment. It's not magic. It's just good design. Learn more at protontext.com. In this episode, our guest Jitendra Zaa is telling a real life story. So, in order to fulfill the business requirements, how he and his team explore different types of asynchronous methods in the large and complex Salesforce work, I personally really really enjoyed this story, and I have definitely learned a lot. I'm sure you will like it too, and we had a one-hour conversation, so the content will fit into two episodes. So this one and the following episodes. With

[01:30] that said. Let's now dive into the show, shall we?

[01:37] Hey everybody, this is Xi Xiao. This is yet another episode of Salesforce Way podcast. Today I'm sitting with a new guest. His name is Jitendra Zaa. Am I right? That is right. Hello, Jitendra. Hello, hello. Welcome to the show. Hey, she. Thanks for inviting me. Sure, sure. I'm very, yeah, very excited to have a talk with you. So, Jitendra, just for the minority part of people, um, they don't know you. Would you like to introduce yourself? Oh yes. Hey everyone. Thanks for having me on this podcast. My name is Jitendra Zaa. I am working on Salesforce ecosystem since the days of S Control, almost the beginning of 2008. So it's more than decade. I am a Salesforce MVP since 2014. Right now, a senior technical architect in IBM. And I have gathered like around twenty three certifications around since last decade, and I am very excited to have a discussion with you on today's topic. Thanks for coming to the show. You are one of the guys who need a lot of effort to maintain the certificates.

[03:06] Yeah. So yeah, certificate. I didn't started my certificate career like four years. My first certificate was in two thousand twelve. Oh, okay. Oh, yeah. So since 2012, I am collecting certificate every six month or every quarter kind of, if just to learn something. Yeah. So certificate is something that keeps me pushing to learn. I still have few certificates pending. No. So I am just targeting. You want to get the next few months. Yeah, yeah. That's a that's a very good way to learn, right? Of course, the Trailhead is there now, but certificate gives you a path. That okay, or kind kind of uh target that in order to get that, this is what you have to understand. You have to read. You have to practice. Yeah. So yeah, for me it's very important. Yeah, as long as you are really learning and you use the certificate scope to really learn stuff and then practice stuff, I think it's always good. Yeah. All right. Let's let's start maybe on our main topic. So I invited you to talk about asynchronous method. I don't even know how to really name them because in in the Lightning platform, the Salesforce platform, we start to have more and more asynchronous ways to handle our executions, right? And I think it's a topic that you are interested in sharing with people, and you think it's important for Salesforce developers to master this skill. That's what I understand.

[04:36] Yes, it is very important to understand the topic, understand when to use what, and why. Why part is very important. Okay. So what I'm going to do is I'm going to drive all of you on a journey. Okay, and this is all real, all real happen on my project. Okay, and that's how one thing worked, another didn't work. If that didn't work, how we go on the next step and so on. Okay, cool. And yeah. Yeah, we'll see about that. Okay, so it's a story time. Yeah, so the topic would be yeah. So the topic would be a whole asynchronous framework, right? How do you use asynchronous in Salesforce? How many governor limits you have? Why you wanna use that? Those kind of the discussions. Okay, cool. How how should we start? Sure. So, here is the problem definition, right? So, in the world of the consultants, architects, and implementation partners, right? It is very, I will say, with me it happens like fifty-fifty percent, fifty-fifty side that I get the new Salesforce org. If you get the new Salesforce Org implementation, your hands are open. You are free to design whatever you want to do. Okay, unfortunately, that's not really case always every time for the implementation partners. Sometimes you inherit Org. Where lot of customizations already happen, it's already in the production. People are already using it. There are end user. So really,

[06:06] you don't have that flexibility to go back in time, change the design, or even if you want something to change, there is a huge impact and analysis you have to do that. What what is going to be ripple effect, good, bad, whatever, right? Sometimes we have time, sometimes we don't have time. So the story is, I inherited kind of similar org. Uh, and just to give you the magnitude of that R, twenty five thousand full Salesforce license, every week six point five million records would be created. It's huge. Yeah, and. Don't ask me why that is not a multi org. Okay, in my if I could have been architect in that project, I would go for the multi org because there are multi multiple business groups there. There is a commerce cloud part, service cloud part, and then there is a field service lightning part. So I was specifically architect for the field service lightning part. Now what happened is we inherited that project. I went doing health analysis of that org, and the total number of developers in that is. It's a huge. It's more than 50 developer I can say, working in the same org at the same time, deploying everything every Wednesday. When I log into the production org and I did my health analysis, I was surprised that almost every team was writing batch Apex, and schedulers. Thinking

[07:36] that if they whatever they are writing in that one single common org, it will schedule at 5 a.m. if they want. It will schedule at 6 a.m. if they want. They did not do the analysis that the Salesforce documentation clearly says. Right, that if you submit something to the Salesforce, there is no SLA, no guarantee that those are synchronous processes are going to execute instantly. And to my surprise, more than half batch of picks I will say was having a batch size of either zero or one.

[08:12] Okay, so that could have been the good qualification for future method. There is so many ways you can get a synchronous Apex working. The reason you want to use a synchronous Apex is you get the fresh governor limit, larger governor limit, right? More CPU time, more heap memory, more SQL limit. Right. So first of all, you should be asynchronous methods for those all the batchables. They should not be the synchronous methods, right? That's the first step. Very, very good point, right? How I validated that it should be synchronous versus asynchronous. If you go the Salesforce governor sheet, governor limit sheet, you will, if any process runs more than five second, so ten second, long running processes, you have a limit that how many. If you let's say have a synchronous process, ten users are performing some operation on that Apex synchronously. Eleventh user will start getting an error. We don't have that option, right? Because you are talking about twenty five thousand users logging in the system, right? And all users are the US users, US time zone. It's not international users, right? And that is a different story, right? They are they are aware many synchronous which we converted to asynchronous, fearing that governor limit. But this is how I at least whatever is in the batch, either remove it from batch or decide is that really a batch. And then there is a existing

[09:43] part, and then there is the implementation part. As a new implementation partner, we had to deliver more for the end user. So that was the challenge: balance existing and then new. Okay, okay, I got it. Okay, so that was my challenge. Okay, uh, what to do? So I thought, okay, why? So you know, now another before going that right, the batch Apex internally has a lot of stage stages. Like first it goes on holding stage, then it goes in queued, then it goes in prepare. So queued means it is waiting for the execution. Preparing means uh. The start method on the batch Apex already started, so there are two three methods in batch Apex: start, execute, and finish. Yes. So when the execute methods are being processed, that's the processed stage, and then either it's a success, aborted, or completed. My experience with Salesforce, I mean, in the there was one implementation I did maybe three four year back, where we submitted a batch Apex, and we sometimes we waited three to four hour. There was no performance issue, nothing, but it will just wait on the queue. There is no guarantee that if you something you. Put in the queue; it will get executed immediately, or maybe the next 30 minutes. It depends on the resources. Okay, so there's no SLA around that. Yeah, another thing is only five batch Apex can execute at a time,

[11:14] and hundred batch Apex can be in a queue called Flex queue. So again, you have a limit. So the team of fifty plus developer, that huge magnitude, or everybody is writing batch Apex. Nobody was checking. Okay, I am submitting a batch Apex. Do I have a hundred limit on my Flex Queue? And let's say if there is no limit left and the Flex Queue, what should I do? I mean, there is a limit of the batch Apex. How much you can submit? Yeah, right. So that was the big problem, right? Even if let's say I hit a Flex Queue, what should be Plan B? Right. What should I do? So, yeah, I kind of blanked out. I didn't have any solution. Yeah. Okay. I started. I started giving thought and thoughts and thoughts. Then I thought, okay, let's move as much as possible into the queue, queueable Apex. So other than batch Apex. We have some more options. We have a scheduler, which is a synchronous. We have platform event. We have a synchronous trigger now. We have future Apex. We have Queueable. Okay, I understand. So you have so many options. Sorry, I understand the curable comes after the batch and the future calls. Right, it's like a newer feature. Very good point. So

[12:44] initially, I think it was—I don't remember. Yeah, batch was for a long time when I started Salesforce. Batch Apex was very early. Then came Future. Okay. Now Future has a sum of limitations. So for an example, there is a lim. You cannot call Future from Future. Okay. You don't get a synchronous job ID from Future. Okay. Ah. Those kind of stuff. So, of course, Salesforce is a great platform because of the ID exchange. Future was the great addition to the platform, but of course, developers and the technical architects they wanted more, and that's where Qable came. So Qable came after Future, and to me, it's a bitween batch and Future. So if I don't want to have a huge processing in the batch Apex, I always prefer Queueable because I get the job ID. I can track where my job is.

[13:45] Hi, here's a quick note. Don't forget to check the show notes of this episode, where you can find short video teasers, the guest contact information, the important complementary materials, and my own learning points, among many other useful information. In addition, I have also started a YouTube channel to share important stuff I learned from my guest. If you are serious about your Salesforce developer journey, you should definitely consider subscribing to this channel. And obviously, the name of this YouTube channel is Salesforce Way. Now let's get back to the show, shall we?

[14:32] So you mentioned the future method has certain limitations. That like it cannot chain itself anymore. If you fire future methods, then there's no way to get the ID and see what's the status. And this the batchable, which is to the maybe the day one method, the asynchronous method, and then the queueable is the new new one, which really sits in between, has all the benefits that either one of them has. Right, correct, correct, correct. I think Qable is since last four, five years. Maybe I'm not wrong. But yeah, it's a pretty new baby. But now we have platform event and all those stuff. I will talk later that how the platform event everything will play together. But yes, okay. My my choice is always Qable. Whatever that is possible using future. Almost all the scenario Qable worked for me, so I always go for Qable. Okay, yeah, I see. Yeah, okay. Okay, so now I have some kind of the variable. Okay, but think now I have another problem. Okay, so let's play on some numbers. So I have uh. So, how many every org has a limit that how many queueable or how many asynchronous Apex? It's overall bucket or umbrella of asynchronous, which includes execute method of batch, which execute future, queueable, everything. Now, every org has a limit of 250k asynchronous

[16:03] jobs per 24 hour. That's the current limit. Why we are recording this podcast? I don't know what's gonna happen in the future. Okay, okay. It's a daily limitation. Let's say yes. That's a twenty four hour limitation. If I talk about my implementation, I have two fifty k. Another limit is either twenty five k, two fifty k. Sorry, per twenty four hour or number of user license multiplied by two hundred. Okay. That's the whatever is higher. Okay. So in my org, the a synchronous limit was twenty five thousand license multiplied by two hundred. Which comes around five millions. Okay, okay. So I have. This is a pretty big org, right? Yes. Pretty big org, and I do have a flexibility. That that's what first came in my mind. Okay, I don't need to worry about the limit, governor limit. That how many a synchronous job can be processed in the twenty four hour. Yeah. Okay, that was the good part for me. I had a huge. Limit there, right? Okay, but there is a slight problem here.

[17:16] In a single transaction, you can fire fifty queryable. Okay, okay, that's the governor limit. Okay, okay. And let's say for an example, batch Apex can only fire one cubable. Okay. Like trigger can fire like 50 cubable. Okay.

[17:37] Future method can fire only one cubable. That's the governor limit we have. Okay. Then the only candidate which fits for me where I can fire more cubable was a scheduler which can fire 50 cubable. Okay. Ah. Okay. Now my problem is my problem is I don't want my developers to write code everywhere. Oh, do I have a limit fired? Querable? What if I don't have a limit? Okay, I wanted them to give a framework. Okay, which they they don't need to worry about the governor limit. Okay, so what I did is I created a custom object, very simple solution. Okay, and I created a set of Apex class. I asked them don't use DataViz dot execute or the standard Salesforce queryable method. Use this Apex class. What that Apex class will do? We have a limit classes in Salesforce where on runtime you can decide how many limits I have left. So, for an example, if someone wants to fire hundred jobs from the execute method of batch, you cannot do it. Because Salesforce will tell you, hey, you have only limit of one. Where the 99 is coming from? It will throw an error. I don't want everywhere try catch. I don't want error handling everywhere in my project. So I created a common platform, common framework. In that framework, I am checking, do I have a governor limit left of cubable? Yes, go ahead. Submit it. If not, save it in the custom object. I will execute it later.

[19:08] Okay, it's a custom solution, right? It's a central pattern that allow all the developers to communicate to it, and it's like a buffer there. If it if it has the buffer, then just fire up. If it doesn't, then save it to the database for the future to execute. Perfect. Okay. Perfect. Perfect. That's a smart one thing about yes. The one thing here, right, is not every piece of functionality can move to the synchronous because there is no guarantee that a synchronous is going to be executed in five second, five minute, or five hour. So developers needs to make sure that whatever they are submitting. Is not around race condition. If you are performing two operation at the same time and two operations are dependent on each other, you don't want to use a synchronous because there is no guarantee what will execute when, right? Yeah, yeah. So there were some guards and guards and rails around that. I mean. In a technical document, I explain them. Hey, when would you want and when you would not want to use framework? Okay. So we solve one problem where don't worry about the governor limit that I am allowed only one queryable or 50 queryable or thousand queryable. Yep. Okay. Because at the end it was the database operation, and you can have 10,000 records saved in Salesforce in a single transaction. So in the worst case scenario, you will submitting 10,000 queueable if you have a limit left. And again, there was another framework, another governor limit. So everything was handled pretty well, like an

[20:38] enterprise project. We had all those scenarios covered. Okay. So storing identifying limit was all set. The bigger problem I had was how to use this five million limit I have. So let's assume that there is no governor limit left. So my job is getting accumulated. In a custom object. Okay, and if you see on my blog, I have a minor article, not the full solution, but the solution where you push in the custom object, you use the dynamic Apex to get it, deserialize it, fire it. That's the core functionality is on my blog. So you're talking about this 24-hour governor limit is reaching to the maximum. Or is it different? Not really. Not no. Not twenty four hours. So basically, I handle that single transaction can fire how many queueable? Yes. Okay. So single transaction can fire either one or fifty or whatever, right? That part was taken care. Yes. The what was my concern was other way around that my org has a five millions of capacity. Yes. Five million job I can fire. Question is how I should fire. How should I go to that magnitude where if my org is giving me five million, how to use that money? The solution. Yeah, I have money, right? Exactly. How to use that?

[22:08] Okay. I have a custom object. I have all the record in my custom object, right? How do I read it? So simplest solution will come. Oh, why not to write a scheduler? So one scheduler will run every one hour. Okay, it will read the custom object jobs, and if the job is all not fired, already fire it. That's the first solution that will come in anyone's mind. So, let's let's do that number. Okay.

[22:42] So let's assume that you have a scheduler which runs every hour, and each scheduler fires fifty job. Okay. Okay, fifty level. The num and you multiply that by 24. The number will come is in one hour. You can only fire 20 1200 jobs. You have a limit of 5 million. Yeah. But out of 5 million, what you can do is just execute 1200. No, that's nothing. No, that will not scale. No. Okay. Then another challenge came is if we can't fire a scheduler every hour, can we fire a scheduler every minute? Okay. Now in Salesforce you cannot have a scheduler. The minimum interval you can have is a one hour. So. If you want to give an effect of firing scheduler every minute, you need to create 60 scheduler job with the running interval one minute, one minute, one minute after that. Yeah, yeah, I remember that. I did it. So even if I go with and if you do the calculation in 24 hour, I can only fire 72,000. Hmm, still it doesn't reach. Doesn't reach that magnitude, like I have a money in my box that I have to use that money. So now another and again I'm not saying this solution is the right solution

[24:12] because then there is a see. What says for start me in my decade of experience is becoming a better programmer. Everywhere there is a governor limit. For me, it is really a good thing because it I challenge myself in that box, how to utilize this, and that is the reason. Today's topic is pretty important, and I will argue all the listeners to wait till end. That how this miracle happened. Okay. Okay. It happened. Okay. It happened. Okay. All right. Yeah. And I hope you are following me. Yes, I'm following you. I'm following you. I want to spend all those monies.

[24:50] So here is the challenge in Salesforce Org, you have a limit of only hundred schedulers can be created. Okay, you are eating up 60 scheduler with only 72,000 job, and you are leaving only 40 scheduler for the whole org. That big org, there is no way this solution will work. Forget about the money in the box. 60 scheduler is no solution. Hmm. Okay. Right. We cannot go in that route. There is no comparison between 72,000 record versus 5 million. Right. Yeah. Okay, let's assume that even if we go with this approach. Okay, let's assume let's forget the 72,000 for the moment and let's forget hundred scheduler limit. The still bigger problem is, like I said in Salesforce, all you can do is submit a job to the queue of queueable or Batch right. There is no guarantee. If you are saying that okay, my scheduler should run at 12:01, and another scheduler is running at 12:02, there is no guarantee which will run first. Right. What if both runs at the same time and I have a single custom object? What if both job picks up the same job, same 50 record? Wouldn't my logic will execute multiple time? So there's a concurrency issue here, right? There is no way for multiple job to know

[26:21] that in this custom object, which job should I pick? First fifty record is you're, another fifty record is you're, another

[26:41] Hmm. Okay. Yeah, that was really very challenging part of this whole solution, right? You understand, right? Five job firing at the same time, all five job wants fifty record. Each job will take top fifty record. Chances are very high, two or three job might pick the same record, and you are doing a multiple processing of the same process. You don't want to do that. Hmm. Correct. Yeah. So the challenge is how to divide the work. It's now it's a worker and thread model, right? You are doing in the weeds now, right? If I were you, I will either ask for help or just quit my job.

[27:20] Unfortunately, you are the principal technical architect, my friend. You can you cannot. It's your responsibility to get it working. Okay, I got it. Yeah, let's move forward for this. Perfect. So then, what happened is, in my college that days, I remember load balancers. How the load balancer works? It's a very simple concept. Okay, you type www. google. com. Your IP address sends a request to which Google server? Load how the load balancer decides that okay I have let's assume hundred server, which server will my request go? Now the load balancer has a very complex algorithm behind the scene, but there is a one simple algorithm load balancers have is they assign request in the buckets. Saying that, okay, for an example, if request they do some mathematics, okay, like divide every request by number four or mod four, whatever the number comes is my bucket. Okay, so there is a concept of bucket in load balancer. Okay, that okay this request should go in bucket one, this request should go on bucket three or five or something. It's algorithm behind. Right. So luckily, I remember my concept from that day. Very simple concept. So in my Apex framework, what I did is I change my the the portion right where I'm submitting a job in my custom object. While submitting my job in the custom object,

[28:50] I have a configuration in my custom metadata saying that my asynchronous framework is going to support how many buckets. I can have a number, 10, 20, 000. It's all up to me. On the basis of that, every job I submit gets a different, different, different bucket. So in our example, we have 60 jobs, 60 scheduler. That means we need a 60 bucket. So whenever anyone is using that framework, I am giving the number to all my record. Hey, you got number one, you got number two, you got number three. That means now each bucket know which record is my record. Okay, I see. Not sure of. Yeah, I understand. So sixty. So scheduler one goes to pick up the fifty record. Give me fifty record where the bucket number is scheduler one bucket. Scheduler ten goes and say, okay, give me the fifty job where my bucket number is, Scheduler ten bucket, and so on. So the scheduler, each scheduler sits behind the bucket, and the bucket gets the task assigned from this proxy, and the proxy has the algorithm to pick up the incoming traffic, right? Correct. Because I already have a custom object. My custom object already have all the payload and everything. All I needed is one extra field on my custom object, saying that this record belongs to job one, job two, job three. So now for me, my challenge was how the job will communicate

[30:20] with each other, saying, hey, you don't process this record and you don't process this record. This is my record. How do I give the accountability of my scheduler on which job? I'm talking about I'm going to hit five million. But out of that five million, how the jobs will know which job is my job? That's where the bucket comes into the picture. Bucket knows that okay, I am only supposed to process bucket one jobs. Sixty schedulers know that I am going to have a sixty buckets in that custom object. Just do the SOQL and get that bucket and top fifty record to process. Okay. So that problem was the problem was how do we make sure that jobs are not executing my jobs repetitively? Yeah, that I resolved by just introducing a simple bucket field. We are ready to rock and roll. Still, we are on seventy two thousand with the invalid. I will say that's the invalid design, kind of right. Okay. Okay. So then, what happened is I tried to find multiple ways that how can I submit more than fifty job. Okay, trust me. In a single transaction, I still have a limit. Okay, there is no way I can go beyond seventy two thousand. Out of five million, I can only grab seventy two thousand money. Yeah, that I did research and research and research. Now this is the heart

[31:50] of the solution. Okay, where things are gonna scale is.

[32:06] Hey, the story still continues. So, see you in the next episode.