Puppeteer Timing, page.goto vs page.click

This is a video about Puppeteer Timing, page.goto vs page.click.

The key part to remember is that page.goto() has a built in “waitForNavigation’ in it. page.click() does not! So if you’re using page.click() remember to use something to wait for your next page to resolve.


Hello there. This is Jordan Hansen. I’m here from cobalt intelligence. And today we’re going to talk a little bit more about puppeteer and puppeteer is great. I use it a lot. It’s a really easy to use, have this browser and JavaScript and node JS. Some of the tougher parts though, with puppet here, have to do with timing when the page is loaded, when it’s not, when the content is loaded when it’s not.

So I wanted to talk about one little trick here and the difference between page dot. Which is when you navigate directly to a URL and when you click a button, sometimes it may be easy just to click the button. And I want to talk about the difference and why go-to is typically better. Okay. If you can, if there’s URL, I think it’s typically better.

And if not, we need to figure some way to handle it. So first things first let’s show go-to, we’re going to here. We’re going to go to the homepage right here, and then we’re going to go to the blog. So let’s say we go over here, we grab this blog URL and then we navigate around. And then once we navigate to the blog, we’re going to get this entry title right here, this right here, see this element.

So we’ll see that we go over here. It opens up the page, it clicks the blog, and then it gets this data right here. Okay. Now let’s try the same thing as if we click the button. So we come over here, we click this blog button and then it navigates us to here. And then we grabbed the thing. And let’s see it run.

Now. There it is. Oh, it failed. Why did it fail that time? Anyone, anyone, anyone raise your hand? Don’t all speak at once. One of you raised, okay. You, yes, that’s correct. Billy in the front. He got it. Right. So when you do go to go to has built in, wait for navigator. Does it say that now it doesn’t say it. Look, see promise, which is off the menu in case multiple, the navigation will resolve with the response of the last redirect.

Okay. So I think this whole thing is built in navigation. Wait, waiting. That is, as in you push it, go go-to and navigate to the page and it waits for it to load. Then it will proceed. I don’t know what w what line is deciding that? I think it’s just like, almost like the spinner. Probably let’s see, I’m curious.

I go here. This has a bunch of Ajax stuff. Yeah. I bet it’s the spare now. So that’s why this works. So go to, it was almost always better because you don’t have to try to time when it’s loaded. The content is loaded. Now the caveat to this is if it’s a modern website and it’s loading things via JavaScript.

So this site, the documentation site, you can see, look the framework, see it’s all there. See all this is there. And then this loads, and after the fact, they’re probably doing that with Ajax, some kind of JavaScript to get the data after it. So the page loads in, then they make a request and pulling the data.

So this wouldn’t matter that much. In that case, you have to have some kind of weight. If you hit click, it doesn’t know is navigating. It could just be, you’re just activates JavaScript or whatever. So click doesn’t wait at all. You hit click and immediately goes down to here and it tries to find this selector.

Now, if it’s really fast website, like if I did the title, it would get this title. Okay. Because that’s like an item that they’re instantly, but any kind of data that thing’s got a load. And that’ll take a little bit of time, so you click it. It’s going to load it. So you gotta do something to wait. So an easy one and the best one that I had, the one I had most luck with wave or selector, and you just put it right in there.

Waiting for a selector is almost always the best. There’s a bunch of other ways for us. You can go wait, page, wait for navigation. This never works as well as you want. It just doesn’t you spot. There’s all these things you can put in there. It never works as well. Now you could do wait for requests. That’s kind of tricky.

You can make sure this request is completed also kind of hard. Sandy Lewis’ response, timeout, or wait for a C there’s a wait for time. That was where you can pass a hard time. That’s just. We’ll make your thing slower, because what happens if you say, wait for time out of five seconds and you paid loads in three seconds, you just lost two seconds every time.

So waivers selector is best. If you can pull it off, sometimes it’s tricky, but let’s try that with wait for a selector and see if we can actually get this thing to work.

Bam. There we go. Cool. There we go. That is it. I wanted to tell the difference between. Before a page I go to and page.click. Okay. Did you do okay. That’s all for today. Thank you.