testing updated version with changes to user agent and headers #92
Conversation
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
|
So i ran this pipeline https://github.com/buildtesters/buildtest/runs/6941983724?check_suite_focus=true and i am getting the same error. I guess if you want to add this link in check https://www.hpcwire.com/2019/01/17/pfizer-hpc-engineer-aims-to-automate-software-stack-testing/ I can also confirm that this page is getting me a 403 so i am inclined to keep this in the exclude list. Let me know if you want me to do any further testing. |
|
Yeah, but the weird thing is that if you add a user agent header, it works for me (at least locally). I'm starting to think that they have explicitly blocked the ip addresses from GitHub actions. I think you might just have to make an exception - I don't have any other ideas for how this could work (or why it's not working) but we aren't doing a good enough job emulating a browser. |
|
Yeah I'll exclude this link that what I have right now. This is not the only link but several others didn't work. Overall this url checker works 95% of the time so it's still pretty good 👍 |
|
This is superceded by #95, which should help with some of these user-agent strings, and for the remainder I'm going to look into selenium soon! |
@shahzebsiddiqui this is a branch for you to test! You should be able to change your action to use this branch, and it will test urlstechie/urlchecker-python#72. Ping me with a link when you can try it out!
Signed-off-by: vsoch vsoch@users.noreply.github.com