Slowly introducing static analysis without changing everything

This episode is brought to you by Mailtrap. More on them later.

Welcome to No Compromises. A peek into the mind of two old web devs who have seen some things. This is Joel.

And this is Aaron.

Recently we have worked on some projects where one of the early things we did was to introduce some tooling and some process to it. And I kind of enjoy that and I thought we could talk about it because I've seen patterns where, let's just take one tool as an example. PHPStan or Larastan, for the people that are using the Laravel-specific flavor. I think there's a subset of people that just don't care about that tool, will never use it. Fine, we're not talking to you. But there's maybe a bigger subset of people that think there's some value in it, maybe have even tried to introduce it to their project but gave up for various reasons. I kind of want to talk about that and some techniques we've used for introducing it to a long-lived project that maybe has some issues with it.

And before we go too far, this is going to be based off PHPStan, but this can be for other tools or even something like Psalm. So if you're using a different version-

Sure.

... so it may not be the exact same steps, but it's the same concept.

Yeah. I've actually never used Psalm on a project, but I'd to think similar mechanisms exist in that tool as exist in PHPStan. But that's a very inclusive note there at the beginning, so I'm glad you had that. Let's talk about PHPStan. Just for those that maybe haven't heard of it. It's a tool that does static analysis and it specifically it's relying on types being present in your code. That's kind of not something that PHP was born with, but something that it has fairly fully embraced in later days here. But it also has this concept of levels. I think 0 through 9 are the levels that are available in PHPStan. And maybe that's the first thing to talk about. Is, well, what level should you use? Aaron, if you're joining a project, you'd probably start it out at Level 9, right?

Oh, well, yeah, 0 through 9. 9 being the most extreme, of course, I'd go there right away. No, it depends on how you're implementing this, but usually the default for that tool is a 5 without setting up anything else. So I'll just run it and see what happens.

Oh, interesting.

But more often than not, that's something that just shows a bunch of stuff that I'm not ready to handle. So I can see that's where people maybe stop.

Yeah, that's a good point. Like, if you don't create a configuration file, it defaults to 5. That sounds right to me. I'm not fact-checking you now, but I believe you.

I think I'm right.

All right. At the moment of recording this, Aaron believes that to be true. I mean, it kind of makes sense though. Like, 5, oh, that's middle of the road, right? It's between 0 and 9. But let's just assume this is a project that started its life in the Laravel 5 days and you've upgraded it. And it's gone from PHP 7 to 8, and maybe you have 50 controllers or 100 controllers, so it's a non-trivial code base. Yes, starting at Level 5, it's probably going to be too much to bite off. One option is, you could say Level 5 it is and then there's this concept in PHPStan of a baseline where you basically just ignore all the errors that you know about right now, and then anything new going forward, it will enforce Level 5. I personally don't like that approach. We've never used it on a project together, but what's your take on that, Aaron? Would you consider that?

I don't like that either, because it sets a bad precedence for, especially junior programmers who maybe don't fully understand what the point of what we're trying to do even is. Which is a valid thing. When you're a junior programmer you don't necessarily understand all the nooks and crannies of what a good project is. You're just building software and so if you have, "Well, in this area because it was broken, we're going to allow it to be broken, but don't put any more broken stuff in there. So learn how to write it new, better way, but don't copy this stuff. By the way, when you start out we want you to copy a lot of stuff, but don't copy this stuff."

Mixed messages is what it's sending, you're right. And honestly, again, I haven't tried this, but if you can't trust your baseline of code and the new code you write is interacting with some of that existing code... I'm not even fully sure you could trust the new things it's checking. It might weaken them or it might force you to fix those things that you tried to ignore in the past anyways. I know people who have used it and who recommend it. Like, if you read the PHPStan docs, they list it as a valid way of going. But for me personally, and I guess Aaron you're with me, I wouldn't do that. What would we do?
Honestly, I'd rather start at Level 0 and get that passing and then move on to Level 1 and then move on to Level 2. And maybe those are weeks apart. Maybe we're going to introduce this at Level 0, but we also got to ship some features this week. So I can't just spend the whole week farting around on getting to Level 5 right away and there's a PR burden of reviewing all this code that you're changing, you're not sure why you're changing it and especially if you don't have good test coverage. There's all these different things but I personally would rather start at a very low level, Level 0, which does catch things and then just slowly ratchet it up from there.

And I'll say the thing that no one is saying. When you do this stuff throughout the life of a project, it's easier for the non-technical client to understand that there's a bug because new features came out. Versus, let's just say you did it all at once. You introduced some bugs, you basically have told them, "Hey, I've done nothing you can see, trust me. Also, your stuff's broken."

That's a good point. You could break code by introducing these types. Absolutely, yes.

So you're like, "I want to do this tool to make it easier on me to break stuff for you."

That's right. So not only for scheduling but also just from a risk point of view, I think it's nice to kind of put these through. But that's not the only different solution that there is, what else could we do?

Okay. So the one that I'm actually using now is sort of a layer on top of that. I will start PHPStan at Level 0, but then I like to use a tangentially related tool called Rector. To me, Rector sort of fits in between a coding standard, you know like we use PHP CS Fixer, it fits in between that and a type checker like PHPStan. It just brings some different rules and different refactorings. But one of the things it adds and this was added fairly recently, I want to say for sure within the last year, but I want to say even within the last six months, is this idea of type coverage.

And it's not coverage in a percent of 0 to 100, but there's different levels, in fact, maybe that's what they call it, type levels. But those go from 0 and I think currently it's 56 or something. And it's way more granular than PHPStan you get 9 levels and that's it. And there might be one thing in a Level 4 that's really hard for you to do, but well, if you're on Level 4, you have to fix it. This is much more granular so you can just go, like, from Level 0 to Level 1 in Rector is way fewer changes. In fact, each level introduces one new rule. And so I really like that approach on top of the PHPStan levels.

Another thing that I have done in the past is also, you can specify files to ignore.

Sure.

Or you can set the specific database directories that you want to be scanned. For example, let's just say you have a really complex system that has modules. Maybe you only want to check the main app and then later on you're going to go through module by module. Because then you could start out with something more like 5 or 4 or something and probably have a smaller load of stuff to do. Again, sometimes you also have those legacy portions of an app where you're just like, "Well, here is 10,000 lines of a service class." Thank goodness they put it into service class but it's 10,000 lines of horribleness. I don't want to even touch this so I'm just going to escape... I'm going to ignore this one. And you still run into that issue we talked about with like, "Oh, is this the good file or not?" But it's another intermediate step to say, "Well, we're trying to do portions of a project at the same time."

Yeah. And you can get as granular as you want, right? You can specify as many paths to either include or exclude. It's not the same thing as the baseline but it it has the same effect, I think.

Yeah, it's a middle ground between introducing another tool, Joel.

I think they go well together, it's like peanut butter and jelly. But no, that's a good point. And one other thing too, just to kind of clarify. Because there are times where while we won't go in on the whole baseline idea, there are times we have chosen to ignore certain types of errors. And again, maybe globally, maybe for a whole directory, or maybe for just one line of a file. Like, "I don't know why you're telling me this isn't working. I know this code is good, I'm just going to use phpstan-ignore-next-line." And now they even added a thing. You can specify a particular type of error to ignore on the next line. That's okay and I think that's pragmatic too. And hopefully, you'd come back and figure those out eventually, but they're not propagating bad behavior in the code either.

Yeah, my experience has been that those don't get fixed. It's just by the time you get there the code is refactored into something else and that original thing you're trying to ignore wouldn't have existed anyway.

Yeah. Or maybe they fixed something in Larastan and now it knows how to detect that. We've bumped into that.

True.

And so just wait it out. Sometimes that's all you need.

The point really is that we just want to go and look at these tools and we can look at all the different options to slowly layer them into our project. It doesn't have to be something that stops us from doing our features, it doesn't have something that derails the whole project. Hopefully, we're not introducing a lot of bugs and we can take it like little steps at a time until we get to the level, which it may not be 9. We don't go to 9 on our projects, or at least on my projects. Joel tried once.

It's an experiment.

Yeah. And just to kind of see kind if anywhere we'd want to land with that stuff.

Excellent.

This episode is sponsored by Mailtrap, an email delivery platform that developers love. An email sending solution with industry best analytics, SMTP and email API, SDKs for major programming languages and 24/7 human support. Try for free at mailtrap.io.

I have to get your opinion on something, Aaron. Do you ever, when you're driving someplace, let's say it's someplace you have been 100 times, okay? I'm going to make it-

So I fall asleep when I drive there, okay.

Okay, good. And I'm going to add one more piece of criteria. It's at least 15 minutes away by car. Do you ever in a situation like that still pull up Google Maps or whatever and use your GPS?

Oh, you're talking to the wrong person.

Why?

You are talking to the absolute wrong person on this. I don't use GPS almost ever.

Really?

Yeah. Well, remember we're old.

Yeah. You get paper maps out on the hood of your car?

Well, no, but I have done that.

Sure.

I know how to fold a paper map. But no, I just don't... It's going to make me sound a weird conspiracy theorist.

Do it.

If the GPS satellites go down, how are you ever going to be able to go around the places? To me, I like to... Well, I don't like this topic because it's making me sound so bad.

That's why I do like it.

My formative years of traveling long distances was around the MapQuest times. So that meant a lot of looking at it, figuring out the best route and printing them out and sort of following those instructions like that. And I just got kind of used to not having a GPS. And then when cars were coming out with GPSs in them, I just was too poor for that. So I didn't have a GPS in my car and by the time I finally had a car that had a GPS in it, they were in the phones and they were using those all the time. But then I had cheap phones and they would always die or they wouldn't pick up the signal. So after all that, because I don't have heated floors so I didn't have a lot of money.

Oh. Yup, here we go.

I just learned to not use a GPS. So the answer to your question, I think I use GPS probably three or four times a year maybe. But for the most part, no. I look at stuff or I might look up a map if I get lost. If I think I'm going somewhere, and I got there but not really then I'll look at the map, but I still won't use GPS. I'm sorry.

So you're a hard no on my question?

Oh, yeah.

And I was hoping to get some support from you because I do. I love pulling it out.

What?

And it's not because I don't know where I'm going. My family all ridicule me, "Joel, you don't know how to get there? We've been there a hundred times." No, you know what? There's a thing called traffic. In Wisconsin, there's road construction and sometimes it'll tell you that if you take a different route, it's going to be 10 minutes faster on this 15-minute drive. So that's my angle but-

That would make it a five-minute drive, which means you're speeding.

I found a wormhole, okay. They have wormholes on Google Maps now. Ah, you've been no help, Aaron. Thanks a lot.

Is your project just limping around and you need it to start running?

That's something we can help with. Just as we talked about in today's episode, we love helping companies get their projects humming in CI, get the tools set up. So, head over to nocompromises.io and see how we can help you.

No Compromises, LLC