Skip to main content

Hello, I am currently learning to use the /v2/hosts/search API endpoint with the optional "fields" parameter. I am successfully getting results but I am having issues specifically with the subfields under "services.http.response.headers", such as "services.http.response.headers.ETag", "services.http.response.headers.Server" and so on.

 

I would expect these fields to work just like "services.port" or "services.service_name" and return a single value in the response, but they don't. If I specify any of these subfields, what I get in the response (as long as the particular service has any http headers) is a large list which seems to contain all values of the headers subfields, even subfields I didnt specify in the request. And the problem is its just a list of values (name of the field is "headers" for each value), so I dont know which value is the Server subfield, which value is the Content_Length subfield and so on (or if these subfields are present in the response at all), so this data isnt very useful. I think they are also in random order.

 

I will add a request below which demonstrates this. Am I doing something wrong or is this a bug? How can I get specific subfields of the HTTP headers with this API? I have not had any similar issues with any other fields

 

https://search.censys.io/api/v2/hosts/search?q=services.http.response.html_title%3A%22Supershell%22&per_page=100&virtual_hosts=EXCLUDE&sort=RELEVANCE&fields=dautonomous_system.asn,services.port,services.service_name,services.http.response.headers.Server,services.http.response.headers.Content_Length,services.http.response.headers.ETag,services.software.uniform_resource_identifier,services.labels]

I agree specifying "services.http.response.headers.Server" and getting every header back does not make sense.

As a workaround I have used the fields “services.http.response.headers.key” and “services.http.response.headers.value.headers” in the past. It returns something like:

"http": {
"response": {
"headers": "
{
"key": "Server",
"value": {
"headers": "
"nginx"
]
}
},
...

You can then filter the header key client side to find your etag/server/etc.


Thanks a lot Pat, works for me 👍


Reply