Skip to main content

I just started using Censys Search and want to export search results which I understand requires using the Censys CLI. So I’ve installed https://github.com/censys/censys-python and created a PAT at https://accounts.censys.io/settings/personal-access-tokens?org=xxx-xxxxxx but when attempting to authenticate I get a “failed to authenticate” error.

 

Your documentation at https://docs.censys.com/reference/get-started is unclear in what the "Censys API ID” is, it sounds like it’s the Organization ID, which I tried to use. I’ve also tried the token name, and my email address. None seem to work.

 

Here is the output:

 

❯ censys config

Censys API ID: xxx-xxxxxx

Censys API Secret: censys_9xxxxxxx7R_JbsxxxxxxxxxxxGS

Do you want color output? ny/n]: y

Failed to authenticate

 

So I’m asking for clarity on what the Censys API ID is supposed to be. And if it is the Org ID, than any other ideas on why it’s failing to authenticate? I already checked RBAC and my user does have “API Access” enabled.

Hey atw, that Censys Python git repo you linked is only compatible with our Legacy Search product. It will not work with Platform API creds, which are what you have.

You should be using the Platform API endpoints provided in the docs page you linked. An example search query cURL including your orgID and PAT would be as follows:

curl --request POST \
--url 'https://api.platform.censys.io/v3/global/search/query?organization_id=1234' \
--header 'accept: application/json' \
--header 'authorization: 5678' \
--header 'content-type: application/json' \
--data '
{
"query": "host.ip: 1.1.1.1/16"
}
'

Additional developer support for Platform in the form of SDKs and a tool similar to Censys Python are coming soon.


Thanks ​@MattK_Censys for the clarification. I have a script now that is working, but I cannot get it to capture more than 500 results from a Search query. On the web search, this query returns over 1,000 results.

I’m doing pagination with a page size of 100 with the API, but it ends after 500 every time. Do you know why this might be?


Same problem, nothing is clear, could have written clear documentation


Hey atw and Senior Shade, Starter tier users are limited to 500 results per search query. This is detailed on the pricing page on the website, but it could have been clearer--I am following up with our team about that. I also just updated the docs to make this clearer as well.

I recommend focusing your queries using filters to get under the 500 result limit. If you’re going to frequently run a query to identify any changes, I recommend using collections.


I do want to use Collections to identify changes, but just trying to get the initial data set into a CSV.
Any hack you can think of to split up the results somehow? I’m thinking maybe filtering by the first 2 IP octets to try to build multiple queries in order to get the desired data, or some stupid hack like that.

Also https://censys.com/resources/pricing doesn’t say anything about an API 500 result limit.


My hack to work around the odd 500 limitation was to create queries that filtered “web.scan_time” with smaller chunks of time, was able to export the ~3,000 results I needed this way.


Reply