Kevin Jones, who joins to talk about his open-source project ApexLink, is a Senior Architect in FinancialForce.
Sponsors
This episode is sponsored by FormAssembly.
FormAssembly and Salesforce eBook – Find out how FormAssembly helps Salesforce customers optimize their data collection.
Main Points
- How does ApexLink work?
- What is ApexAssist VSCode extension?
- Future roadmap for ApexLink and ApexAssist – the local execution
- Kevin’s daily work in FinancialForce
Links
Video Teaser
Contact Me
If you enjoy this podcast, please reach me out to let me know your thoughts or simply say Hello. My (Xi’s) contact info:
Take care, until next time :)!
Transcript
AI-generated, unedited.
[00:01] I'd like to welcome our sponsor, Form Assembly. Form Assembly's all-in-one web form platform lets you create forms for just about any use case. From contact forms to donation forms, all while taking advantage of useful features such as notifications, e-signatures, and more. Not only that, but you can also connect data to systems you already use. FormAssembly integrates with Salesforce, Podio, PayPal, and many other common solutions. You can find out how form assemblies help Salesforce customers optimize their data connection in a free ebook that we've linked in today's show notes. Whatever your data collection needs are, you can be sure that Form Assembly keeps your data secure with encryption at rest and in transit on all plans. Plus compliance with GDPR, CCPA, and more regulations. At the end of the day, formas simply helps you save time, money, and effort while getting the maximum benefit out of the data you collect. And I'd remind you, when you support our sponsors, you support the show.
[01:23] Hi. This is a continuation from the last episode.
[01:31] So
[01:34] now it's it's getting much clearer for me. So the first step is really just to. Parse the things out and you know put them onto the table. And the second step, which is much more difficult, as I see, it's really pull the strings to see where the dependencies. Really, yeah, yeah, yeah. That's a good way of thinking about it. Yeah, if you get everything on the table, then you can see what the problem is, and then you can start work. Yeah. Okay, then what what is the output from this phase? Did you get some certain map or or or diagram out of this? So there's there's really two the the headline output if you like the thing that most people see is the issue log as it's termed in in the tooling, which is really just a set of diagnostic messages that says you know you reference this class here that class doesn't exist or you try to call this function and that doesn't exist you know that kind of. Pretty basic level of of error messaging. There is a secondary result here, which is in in some ways is more important for ongoing tooling. Which is part of this validation second pass I've been talking about. We build a dependency graph between the components. There's actually two dependency graphs in it. There's one which is built at the class level, so the outer class level. So it says this outer class depends on this one. But internally, there's a more detailed version that says this code block. Inside this method, say is referencing that function over there, so it
[03:05] it ties it up at a much more detailed level, which is very useful for when you want to start doing more detailed analysis. So, on one of the example analysis I've been playing with is. Showing unused methods and fields, which is often a problem in large code bases where people just change and forget that actually this field is no longer used, and that's done through the dependency analysis. Essentially, we do the the full validation of the package, and then we go through through the classes and say, "Okay, let's look at all of these fields and check that somebody's referencing them. And if nobody's referencing them, they're probably not in use." And that's how it kind of works. So t here is this kind of graph hidden underneath the surface, which is valuable for second level analysis, really, or further analysis. When I use, for example, Visual Studio Code to write the JavaScript, the LinkedIn tool once I declare, for example, stream and I never use it in the class, then it's grayed out in color, which tells me it's never used. So this is similar to what you're doing, right? Yeah, yeah, very similar. I think you you. You've got to have a reasonably high level of confidence that you're finding all references, which is why I'm using it as a sample problem to make sure that I'm, you know, not missing anything. And then, if you know that nobody's referencing it, then you can just say, "Yeah, you're not in use." There are other techniques you can use to find similar, but this is the most exact. And obviously,
[04:35] reducing false positive rates in tooling is really, you know, very important. So, given this is a pre-exact model, it's quite quite handy. I see. So that's the output of this phase. Let's say, how much time would it take if I have a regular size? Manage the package. I want to run your tool. How long does it take to get the output? So it does vary a little based on something we haven't talked about yet. It has a cache. I'll ignore that for the moment. On the packages I use, we're typically looking at maybe thirty seconds for an analysis. Obviously, everybody's hardware is a little different. You know, if that's on kind of pretty modern hardware, if you've got something a little older, it'll take a bit longer. And that's that's to run through all of the metadata, do the paths, and then run this validation pass to build the graph and report any issues out. The cache is something I've added fairly recently, which is an on disk cache. So, after the validation phase, after we've done all of this work, I can now flush the parsed version onto disk, and then next time you analyze that project or something pretty similar, it will load the majority of that metadata straight off the cache. And in those cases, it's kind of I typically see maybe ten seconds as a load time or analysis time, so it's kind of about a third of the cost
[06:05] of the original version. Okay, it's good we have a cache, right? So we don't need to run it all the time. Yes, yeah, it is. There's various versions of that. I I think the one we've got is quite good in that. It's if you're working, say you've got two different directories. It's the same cache. So as a developer, you know. You often move around different directories which have the same cause. So I'm coding. So the cache works. It's not in the directory. It's in your home directory. So the cache will work on any version of you that you've got. And if you've got a branch out, which maybe has a slight change from the one that originally, it still works. So I kind of like the cache. That it's very flexible and it's been a fairly recent addition. It's taken quite a bit of effort to get in, but it does work really well. That's cool. That's cool. I understand. This is all the step, right, from the high level point of view. The final output is that map that's cached into your disk. Yeah. So the dependencies are, you know, they they end up on the cache. They're also in memory. If you want to interrogate them dynamically, you know, we can bring this back into memory now quite quickly. So you can run other analysis. I think maybe what's important here to understand is I very much see Apex Link as being. A base level package. It's quite complex internally, and what
[07:35] you don't want to do is kind of throw lots of different analysis types into that kind of code base. So, if I want to do an analysis, I build it as a separate piece of logic. So I use that to build up the graph, so I can understand the codebase, and then I write the analysis as a separate piece, because each analysis tends to be slightly different. The one I haven't done separately is the one that's currently out there, the kind of unused. That's kind of built in because it was just you know quick and easy for me to do it that way. But largely, it's better not to try and build all of your analysis into your core because the core just gets too complex. So to give examples of things that we've looked at in the past, what one of them is a bit of an odd one as an example, but one of the questions I was asked a while ago was: if you went to a Visual Force page, could you tell me what labels might appear on that Visual Force page? And and people think oh well that's easy I just look at the labels on the page that are in use but of course a lot of the labels that would appear on a page come through exceptions you know certainly in our packages you know it's the exception messages which are coming off labels so actually what you need to do is start at the page walk from the page into the controller which is in Apex. Then walk all of the methods that that's calling, and have a look at each of these methods and find out what labels they're using, and then you can get a complete set of all the labels that might appear. Now that
[09:06] was a very specific analysis somebody was interested in doing, and and I kind of played with it and implemented it, but it's implemented as something separate from the core library. You know, it's just a layer because it's a very specific kind of problem that sits over the top. I see, I see, I see. You have another tool called Apex Assist. Right, yeah. Is that the one you use to analyze the code? Yeah. So I did that. The I did. I wanted to have an extension because one of the problems of the library is it produces like command line output of issues. And as nice as they they are, you still have to then navigate to the file and find the line number and do all of the mechanics of that, which is kind of slow if you're checking lots of issues. So putting it inside VS Code. It makes that process a lot quicker, a lot more natural. So, basically, I created it just really to enable me to do that and and see how it worked inside a VS Code environment. There is a little bit more to it, which is if we go back to our discussion around using Scala. T here is a Scala JS sub project which allows translation of Scala into JavaScript. Which, when I first read about this, oh, this is probably not going to work very well. So I had a bit of a play with it, and it worked really well. So I thought I'll carry on doing this, and before I knew it. Yeah, yeah. I think most of modern languages nowadays
[10:36] has the .js to combine JavaScript. Yeah, everybody needs one of those. It's mandatory these days. Yeah. So I carried on and and and I kind of ended up having the whole thing running JavaScript. I'm like, that was easy. So, so the Apex Assist extension is kind of a test bed, but it's also entirely in JavaScript to see kind of see how that works. There is some practical reason for doing that. I think. You probably know getting everybody to install Java at the right version and have all the paths set up is a bit awkward and often error-prone, and that's not what you really want with an extension. If you don't have to do that, you know, so having it in JavaScript means it installs much easier, and the support issues are going to be lower on there. The downside, as I think I've mentioned a couple of times on Twitter, is you know performance isn't on a par. You know it really isn't. So it's about three times slower than using the JVM version. But okay, but if I'm not a hardcore developer, I just want to test how this tool can work. You know, start from the output part. You know, check the the graph you generated. So I can use this Apex Assist, which is a Visual Studio Code plugin. Yeah, I can install it from Visual Studio. Okay, yeah, it's on the marketplace. Yep. Yeah, I will put the the link for both of these two tools into our show notes for for people to check. Okay. That'd be great. What what's your future plan? I understand you have achieved such amazing. So, what are you going to do in the
[12:06] future? Well, I've got quite a few. We've mentioned one before. I'm I'm interested in generics templates and you know different ways of dealing with language. We've looked a bit, or at least I've looked a bit at local execution, which probably is a bit shocking as an idea. So, if you understand the package and you understand the language, in can I execute the code in there? And the answer is yes, I can. Then the question is, why would you want to do that? You know, Salesforce do all that. Well, actually, it turns out there's a pretty good reason. If you go back to issues we have around running large scale test suites, this has got better over recent years for us. So the pressure on this is not as bad in in financial force. But it used to be really, really expensive, and even now, you know, the ability to do test-driven development on the platform is quite restricted. You know, it takes a long time to run tests. We tend to run them in our CI pipeline and rely on that to provide feedback. As you know, as a developer, when you when you say it's restricted, do you mean you need to send the local Apex file to the cloud and then run? Is that the main restriction? Yeah, it's it's the volume of tests that you have to run and how long they take. So I'll give you the parallel here. The full test suite for Apex Link runs in around about eighteen seconds on my
[13:37] laptop. That's the whole test suite, which is about 800 tests. If I was to run that on Salesforce, I would be probably looking, depend on whether you run it in parallel or sequential. You know, I'm looking at at least five minutes. You know, it's an order of magnitude different. And in a TDD world, why is why is there such a huge difference? The the platform is not optimized to run this unit test Apex tests. I think there's a number of factors you probably point to. Obviously, in in the Salesforce world, we're you know running tests against a live database often and. The level of mocking that you've used in your tests influences the performance greatly. Parallel testing on on the platform has some issues, although we use use that quite a lot. Parallel unit testing it's very very beneficial if you can. Get it to work reliably. You get this unlocked error or something like that. Yeah, unable to lock row is a pretty common problem. So you have to kind of work around that. Really, just accept that that's what happens.
[14:50] And then I think just execution speed is quite different locally from on the platform, isn't it? I think we all know that the, you know, share CPU performance is quite different as to how that works. So you end up with quite a pragmatic difference, don't you, in local test speed versus what you can achieve on the platform. And and that discourages people, discourages developers from ever really trying to do TDD. I I think some do try it. You know, some really want to get into it. They understand the benefits of doing that, but it's. It's a bit like the refactoring problem I mentioned earlier. You know, it's just not something that's easily achievable today. You have to be pretty persistent to get there. So local execution for me really is a matter of saying, well, actually, could we do something around that area that would improve the test throughput? So we could test it on our local machines at much higher performance. You know, so I want to see an order of magnitude is not enough. I want to see two to three orders of magnitude quicker on the test infrastructure, really, to get into that TDD space. I think. I've done a little bit of work around that area, but I still think you know it's still a future for me. It's out there. It looks like you could do it. We've got we're almost getting to the point where we've got enough tooling to do that kind of work. My skill is it's totally beyond my skill level, so I can kind of thinking about it, but
[16:21] I don't know how to achieve it. It's really up to your guys, like the senior guys, really. And it's a huge benefit for us as a developer. You know, we want to run it locally whenever possible. Yeah. Yeah, I think there are quite a few, quite quite a lot of exciting things you can do off the back of this kind of tooling. It's just you know people's interest and whether they take it. I put an awful lot of kind of. Personal effort, if you like, into into trying to get this tooling to a state, but there is a limit to where I can run to. So, people are interested in this stuff. I would encourage them to kind of reach out, and I'll try and help them as much as I can to bootstrap on, you know, whatever it is you are passionate about. But, you know, Apex Link has to remain relatively small, just because of the you know limited resource, of course. I see. Thanks a lot. Yeah, I will put your contact information. You did have a repo, and even shout out in the Twitter as well. Just you know, for the people, the geeks who really want to join this. Yeah, huge benefit. If you got a burning desire to see something on the platform, let's make it happen. Yeah, um, that's all I want to talk about the open source tools you created. So let's now shift a bit back to your role in the financial force. It's a huge company. I know. What's your daily work? What's your role in the company? Oh yeah, so that is a big shift. So, so I guess I I mentioned earlier. So I work for the chief architect.
[17:51] So there's a little bit of history there that I guess people find interesting. So when I joined, Andy Fossett was the CTO of Financial Force. So I worked for Andy for I think it was a couple of years before he moved off and joined Salesforce. And since then, we've kind of moved to a model where we have a chief architect instead of a CTO. So I work for the the chief architect inside the company. We kind of have a model these days of saying we don't like too much centralized architecture. I think this is probably different from a lot of smaller ISVs. We're just too big. To try and centralise for for one small group to like be the architects, if you like. So what we do is we have architects on all of those core products that I was talking about earlier. So they all have dedicated architects. They're much closer to the product. They can work with the product managers and with the engineering managers more closely, and and of course they work on the products as well. You know they're developers on the products. So in terms of what I do, I guess. Partly, I'm supporting that team of product architects and trying to make sure that they're being successful and they have the right tools and capabilities around them. One of the interesting things about decentralisation, from my point of view, is that you then got to look back at it and say, "Well, it's okay, everybody doing their own thing, but occasionally we want consistency."
[19:21] So we need to share what we learn about the platform between us. You know that's the best way for us to learn. Is so we communicate quite a lot between the architects, and also where it kind of makes sense is to try and make sure that the products are consistent. Unless there's good reason not to, because there's a lot of discussion around. You know, this product would like to do it this way, and this product would like to do it another way. And is that is it right that they do different things, or should we try and make sure that they're doing the same thing or find common ground? So that's a big part of kind of my day to day. I think there are kind of another couple of pieces I do. One is, I guess, what people would think as big architecture. So this is the big strategy stuff, as opposed to the kind of more detailed product work. A lot of that, really, I think, is about trying to make sure that the architecture is fitting what the customers really need, and if it's not, understanding how we might change things around. So, to give a more concrete example, there's there's a lot of interest around, say, scalability of products. You know, that's a big architecture concern. Because often it goes into strategy about how the product has implemented a certain feature, but it also goes into how the platform supports that kind of feature. You need to have a good understanding of both sets and try and understand whether we're making the right decisions in certain areas of the product or not. And they tend
[20:51] to lead into, I guess, what you'd think of as kind of. Big expense items, as in they require a lot of team effort to perhaps change a way that we're doing something, and a lot of discussion with product managers and engineering managers to make sure we're all making the right decisions. There is another aspect to to what I do, which is I have a I run a team as well who take on largely what the what you might think as the difficult or hard problems that that come up. So these are they're generally architectural in nature, but they're also customer problems or some mix of those. You know, it's often hard just to say one problem is one thing, isn't it? It's You know, it's something that has an architectural element. You know, a serious design element, but often it's driven by some customer need. So that team works on those kinds of issues as well. And does quite a lot of core research into Salesforce. So if you see me talking about or hear me talking about some feature regards to Salesforce, which is a bit kind of esoteric, usually it's coming out of that work because we need to have a level of understanding of Salesforce, which is pretty deep at times, in order to try and make the right options available for the product teams to choose from. I see. Actually, such a team is like a R and D team in financial force in the company, right? Yeah, really drilled into the platform features and to see something if it works and often some edge cases as well. I'd assume.
[22:22] Yeah, I was slightly laughing because in fact they they used to be called the they are currently called the research team inside financial force. They're going to have a slight rename, but yeah, you've you've nailed it in one. Yeah, yeah. So they do almost fundamental research into how the platform operates at times. You know. Which is really actually it's a really interesting area, very challenging in a cloud infrastructure because you know clearly from our point of view, Salesforce is much a black box. You know, we can't see into the internal mechanics, and often we're trying to work out what what is it about the internal mechanics that we don't understand. And that's really quite challenging, you know, particularly around performance, scalability, you know, the things that, you know, for for the right reasons, Salesforce don't really want to give us a lot of information about, you know, they don't want us to become dependent upon their internal implementation, but at the same time, you know, there are key bits of information that if we don't understand, we can make the wrong choices. So there's quite a lot of work around that area. All right, Kevin. That's all I want to you know pick your brain from, and yeah. So thanks a lot for sharing your knowledge. So you. Great chatting with you. Thank you very much. See you next time.
[23:41] Yeah.
