When validation can protect your app's performance
Welcome to No Compromises. A peek into the mind of two old web devs who have seen some things. This is Joel.
And this is Aaron. Joel, what is the limit of the pagination in Laravel? The default pagination.
I didn't know this was going to be a quiz. 10.
10 or 15, or something. I don't know. I think it's 15, but it feels like it should be 10. But the fact is-
I have 10 fingers so.
I guess that makes sense. So pagination, it has a limit, we can go through our pages. Why do we even have pagination as a pattern? Like, what are some of the reasons why we use that?
I mean, I think, for me, the number one reason that comes to mind is performance, Like, I'm thinking of a large data set. Maybe you have 150,000 rows, that would bog the system down. That's one.
Right. And then you have a user interface and all these different things. I'm glad that you brought up the performance one because that was the one I was going to focus on real quick.
Good.
So I guess I'm going to try to interpret what you said and make an assumption.
That is your job. You're the Joel translator.
And you correct me if I'm wrong.
Okay.
So you're saying that pagination is helpful because we can put a limit on the results maybe that we retrieve. So for performance, we either over the wire transmitting it or the query, like limiting the amount of information we retrieve or how much of a query we have to do. That's the thing you're referring to when you say performance, right? Something to do with that.
Yeah. You're hinting at, there's a couple different layers to it. In my mind, I'll admit I was actually thinking of the time it would take to run the query. So if it's a row, for example, that's not just a straight select. It's a select with a join that filters this and has a relationship existence check. And like all that stuff where if you're doing it on 10 records it's instant, but if you run that across the whole database all of a sudden it's really slow. Or you're sorting, or you're filtering, like those sorts of things.
Now, I know I haven't been telling you kind of where this podcast topic's going to go and I know that it kind of feels like a trap but you fell into it perfectly.
Oh, come on.
So this idea that we can use pagination to sort of limit the workload, the intensity of the query, all these different things. Then my question is, what's the limit for the high level of the per page?
Ooh.
Well, it sounds like that's kind of built into this assumption. Is like, hey, there's pagination, so there must be a limit. And because of that limit, we have less workload so there must be a default. But what if someone understands that that's the reason why you're doing pagination and is a little evil and wants to raise that limit up? What's the Laravel limit on how high it can go?
I don't think there is one.
Exactly, I don't think there is either. So that's kind of what I wanted to talk about today, which was we talk a lot about validation and testing and stuff, but there's other places you can do validation that aren't necessarily a sort of area that you think about but is somewhere that needs to be covered. And one of those is actually GET requests. And that's not something you really think about when you think about form validation or stuff like that. But you are doing something based off user input and that user input should be validated and appropriately relevant to your application.
I 100% agree. And I could say, correct me if I'm wrong, I don't think we've ever joined a project that validated the pagination size. Like, I've never seen it done other than us doing it. And just one aside, just to make sure we're not being critical of Laravel. Because I don't think... Do you think there is a sane upper limit that it could impose that would actually make sense across all applications? It feels kind of hard.
Well, yeah, I think that's one of those things where it's like, well, until there's a problem we don't necessarily put something in there. So it's not necessarily criticism, but there is something that you should understand the tool that you're using.
Yep, agreed.
And this is one of those things, again, where we say there's a lot of cool functionality inside of Laravel but people can abuse it. So, for example, fillable or guarded on your Eloquent, don't just put a star in there. Like, stop that. But you could, and I'm glad that Laravel gives you the ability to do that, but don't do that. So it's same sort of concept here, I think.
I agree, yeah.
So what we do in situations like this is we need to make sure that we have some sort of validation to make sure if we're kind of... I guess there's two reasons. One, if we're even depending on pagination to be our sort of safety value for our performance issues. And two, like I said, when it comes to user input, we should always be validating it anyway because I just don't know all the things that can happen. If I expect from a user that it should be a number or nothing, and that number should be between these values, that's the only thing I ever want to have from them. Because I don't know what's going to happen in the future. And here's a great thing. Here's where being a salty seasoned professional helps me, is, back in the day we didn't even think about UTF-8 that much. So you would have windows type character sets, and things like that, and people would send in UTF-8 or an emoji or something like that. And it would just bork everything up.
That's right.
So you're like, "Well, I didn't even think about that." So when you're thinking about validation, you don't know what's going to come down in the future. Maybe there's a bug in a browser that always, for whatever reason, Chrome, puts out a bug that says per page equals a thousand. What would that do to your app? I would hope it would stop it versus just take it all down.
Yeah. You mentioned could be a bug, could be somebody malicious, could be somebody just messing around. And I thought even related to this, let's say you have zero performance issues, Say, maybe your database has a million rows and you could load them all and it would still come back in like 200 milliseconds. There's also like, I don't know if data privacy is the right word, but like do you really want to give somebody a whole dump of a table? Or maybe you want to at least make it a little harder for them to scrape your site and pull all that data out? There's all these good reasons to enforce it but, yeah, it's... I think GET requests in general. I rarely see validation done on them but this is just one specific example where I think it's super important.
So how we might do that is usually on our... If we're doing a resourceful controller or something like that, you have your index method so we might make like a index form request. You know you have your store and your update form requests that you might use for validation, we'll make one for our GET request too. And in there it'll define rules which get processed automatically because Laravel is like, "I don't care what method you're using, I'm going to process this form validation." So you can put in stuff like the page, the upper limits for your page, and per page. I like to put some just little sanity levels on there as well as I don't let negative numbers come in because I just... It's not a useful thing to have happen and I don't know what would happen. I mean, I know what would happen, nothing bad will happen. But I don't always know that, right? So we put that validation on there. There's one small caveat that I'll mention though that has bit me a couple times. And this is one of those things where it's like, I don't care if something goes wrong for someone who's abusing my site, except for maybe I do a little bit. And then especially if it's for me when I'm trying to develop it and it goes wrong and I can't figure it out. So, when those form requests fail their validation, the Laravel validation will try to go to the previous page.
Nice.
So if you're going to this page, it'll just go into some sort of loop and that could be a problem. So you can customize that on that form request on, like where I want to go, so you would just kind of tell it to go to itself again without any GET requests.
Okay. And it's mainly an issue if you are validating a GET request because, like a post the way you're coming from is where you'd want to go back to, but with a GET maybe not, right?
Yeah. With a post you're posting from somewhere, you're really never coming from nowhere. Or if you are, there's like an intended route then.
That's a good subtlety, I'm glad you brought that up. I want to comment one more on one more thing. You mentioned negative numbers, like, nothing bad happens if somebody says page size or page number -1. Well, nothing bad happens right now. Again, maybe Laravel is handling all that for us now, but maybe down the road, there's a bug there. I don't know what it would be, but this concept of stopping bad data at the edges of your app is just like a good general principle. So why wouldn't you? Like, even if it doesn't actually matter right now, it might matter in the future, it's easy to do and it just structurally documents what you're expecting if somebody looks at that code later. So I think there's all sorts of good reasons.
The easiest way for a chicken to give birth is with a C-section.
What on earth are you talking about?
I know, right? Isn't that just a weird sentence? There's something weird about that. Like, there's so many things wrong with that.
Chickens don't give birth. That's like... you lost me immediately.
I know. I know, exactly. So the reason I'm bring this up is because I'm really kind of getting nervous about what's happening with AI. And I'm not going to be like an old scary man here. But remember Google bombing?
I do.
When we used to do that. Not we, some people used to do that.
I never did.
You would go and you'd develop a term or something and you could go and seed that over everything. And then when people would search that term, it would look like that was a natural response to that. And that was the bomb and Google is like you said a weird term, and then it now responds to that.
That's right.
And some people did it for famous people for a while until Google was able to fix that. Now, that's kind of my concern with this generative AI stuff now that we're seeing too, is it's a bit smarter than that. But what if people that want to mess with it are just smarter too? So, I crafted that sentence specifically because I wanted to mess with AI. So I wonder if it's, follow me through, because I'm thinking about how AI works. Generative, it doesn't know but it starts to build patterns. So when you start to mess with the patterns, maybe you can convince it something else, so, all right. It knows what a chicken is, all right. Easiest way for a chicken to give birth? What? Chickens don't give birth. So already AI is spinning out of control like you are. This is false information, I'm going to start to throw this out. But then I continue. I'm like, "No. It give birth is a C-section." So I'm reinforcing that, yeah, I'm doubling down on that. That they can give birth. And pretty soon it's like, "Well, I guess this is a good pattern because if it was a bad pattern, this human detail would've just given up by now." So I'm training that AI now that it's going to have this weird sort of thought process in its head. Before you know it, when we start asking for eggs it's going to give all kinds of weird suggestions in the grocery store. You're welcome. You seem to know a lot about validation. Like, you were even sort of almost wanting to talk over me because you knew so much about form requests. Why is that?
Oh, I get so excited. We wrote a book on it, Aaron. So, if you listened to this and you liked our takes on validation, head to masteringlaravel.io, click on validation, and check out that book.