On the hunt for data leaks: Elastic(search²)
[BSides Athens 2024 tool presentation slides.]
Since October 2023, I've discovered more than 10M PII of people and helped many companies worldwide secure sensitive information. The companies range from e-shops and hospitals to human resource consulting agencies. They all had one catastrophic thing in common: misconfigured Elasticsearch hosts.
Elasticsearch is a storage and search engine for analytics. It is often also used as a database instead of other DBMS. It is particularly common for e-shops to use Elasticsearch as a database, storing order and shipment details, which include customer PII such as
- Full names
- ID numbers and photos
- Physical addresses
- Phone numbers
- E-mail addresses
etc. As with any technology used, it is not uncommon for misconfigurations to occur, such as missing authentication or accidental exposure of internal hosts on the Internet. The goal is to go after such hosts, identify them, and report them to the organizations (when it's possible to find them).
Statistics
Out of the discovered exposed hosts of significance (meaning that the exposure causes an actual leak of PII), here are some results:
- Around 50% of the hosts are in Asia, specifically China, India, and Bangladesh—the rest 50% are from Europe. The US was not covered broadly. Yet.
- Around 20% of the vulnerable organizations are e-shops containing order tracking details. The remaining 80% vary from governmental bodies to other sorts of companies.
- More than 90% of the hosts that belong to governmental bodies are from China. Only one discovered host belonging to a government was not from China but from a Bangladeshi SIGINT agency.
Elastic(search²): host aggregation and analysis
This work and research wouldn't be possible without some automation. A CLI script would be quick and to the point but I decided to build a web tool with analysis capabilities, and a nice presentation of the results. The tool is called Elastic(search²).
It is as easy as entering your Shodan API key, choosing the country, and hitting enter.
The tool has the ability to keep track of the already-discovered hosts and update you on the new ones. There is keyword analysis that identifies strings and words on indices' names that are potential leaks and also an on-demand keyword analysis on indices to identify strings such as JWT, "password", "fullname" etc. Both keyword lists can be adjusted to add more words. There's also a cleanup function, i.e. to get rid of stored hosts that are no longer responsive. Please, try it out!
Case study: PII of Ukrainian citizens
One simple case of leaked PII is the discovery of the personal information of more than 1 million citizens. The leak came from the Ukrainian human resource consulting GRC.UA. The PII included
- full names
- email addresses
- phone numbers
- professions
- cities of residence
After the discovery, GRC.UA was contacted and the issue was resolved in 1-2 hours. During a war, citizen, firm, and government information plays a crucial role.
Case study: From JWT leak to account takeover and pwning all the users.
This should (and will) be covered in a separate post. The discovery of a misconfigured Elasticsearch host that leaked the JWT of some users of an Android application used for paying utility bills, lead to accessing those users' accounts. After researching the application a bit more, it was possible to takeover any of the 10K user accounts of that application. But more on that later...
Future steps
The next step is to add more search engines for the tool to grab hosts from. Another idea is to add more customization in your search such as the size of the hosts you want to discover (GB or MB, currently set to GB) and maybe SSL-specific searches. The SSL tag on Shodan helped me discover great attack surface when bug bounty hunting.
Another step is to turn this tool into a script that can run automatically and either include it in your automation or run it periodically to get new hosts daily.
Let me know your ideas! Feel free to reach out or open an issue.