Hi,
I have a query which return 200K hosts. I need to download all.
if I use the following code, after 300 query rate limit will stop the script.
query = h.search ("SEARCH TERM", per_page=100, pages=-1)Also I used another approach, which download lots of repetitive data
for page in range(1, 100):
query = h.search( 'services.port: 80 AND services.tls.certificates: * AND services.http.response.status_code: 403', per_page=100, page=page)
any IDEA how to download all data without problem.
Hey Mari, I checked with our team and I think you need to add a time.sleep(1) function to your code. Here’s a bit of info about that: https://www.digitalocean.com/community/tutorials/python-time-sleep
Let me know if that helps.
I have sleep(2) in my code after each API call, but I don’t know why I got redundant data. I lose half of my quota because of this.
Hey again Mari, to help further we’d need to see the full code you’re using as well as examples of the redundant data. Please feel free to open a support ticket or message me directly with that info.
thanks, just opened a ticket.
Already have an account? Login
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
