Hi,
I have a query which return 200K hosts. I need to download all.
if I use the following code, after 300 query rate limit will stop the script.
query = h.search ("SEARCH TERM", per_page=100, pages=-1)Also I used another approach, which download lots of repetitive data
for page in range(1, 100):
query = h.search( 'services.port: 80 AND services.tls.certificates: * AND services.http.response.status_code: 403', per_page=100, page=page)
any IDEA how to download all data without problem.
+2Hey Mari, I checked with our team and I think you need to add a time.sleep(1) function to your code. Here’s a bit of info about that: https://www.digitalocean.com/community/tutorials/python-time-sleep
Let me know if that helps.
I have sleep(2) in my code after each API call, but I don’t know why I got redundant data. I lose half of my quota because of this.
+2Hey again Mari, to help further we’d need to see the full code you’re using as well as examples of the redundant data. Please feel free to open a support ticket or message me directly with that info.
Already have an account? Login
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.