Atlanta's Hartsfield-Jackson International Airport is the busiest in the world, serving over 93 million passengers to and from this international destination last year. There are many reasons to visit Atlanta, such as seeing the home of Coca-Cola, visiting the birthplace of Dr. Martin Luther King, Jr., or taking in a Braves game. But none of those reasons were why over 500 security professionals gathered at the Kennesaw State University Center. They gathered to share their knowledge and experiences defending their organizations and teams at BSides ATL 2023.
In the opening remarks, we learned this was the largest free BSides in the south, welcoming just over 500 attendees, vendors, speakers, and volunteers. Folks had a lot of programming options, including four speaking tracks, a lock pick village, and a network hacking 'king of the hill' capture the flag.
Here are just a few highlights from this community-led event.
Separating your security work and your personal life
In her opening keynote, Sherrod DeGrippo, Director of Threat Intelligence Strategy at Microsoft, began by talking about logging and ended with a call to action for all security professionals to guard their happiness. She said we are threat hunters professionally and are trained to use tools like Wireshark, Splunk, and Cuckoo Sandbox to examine traces left behind by malicious actors. We are experts at finding broken and insecure locks and system flaws that hackers with bad intentions can exploit. Psychologically, this trains our brains to reward us with an emotional high when we find an issue. As she summed it up, we get to a place where we "find something bad, we feel good." And there is a lot of bad to find.
While this 'detect and reward' cycle is useful professionally, it can have serious negative effects on our home and personal lives. She explained that if you start applying your hunting techniques to people in your private life, you will likely have some very bad outcomes. She reminded the audience that Zero Trust is only for systems and machines. With people, we should generally apply "trust, but verify." Assuming bad intent from our friends and family leaves us with paranoia and mistrust of everyone.
Keynote at #bsidesatl kicking things off with Sherrod DeGrippo, Director of Threat Intelligence Strategy at Microsoft
The insecurity in Web3.0
In his talk "Unchained: Unraveling the Unconventional Security Threats in Web3," Nick Kammerdiener, Senior DevSecOps Engineer at Hypergiant, asked us to think about the problems presented in blockchain and what we can do about them. There are a lot of use cases for Web3.0, also commonly described as Web3 or blockchain technology, from cryptocurrency to public records where individuals can control their own data. There has been a lot of hype around this way of building decentralized systems, but like AI, many projects come from conversations with non-technical leaders that sound like, "We don't know what Web3 is, but we know we need it."
This has led to a few issues, according to Nick. He spelled out three areas of concern for Web3.0 security.
Insecure development practices
Known issues with common services
Network attacks
First, developers are struggling to keep up with the tech, leading to a lot of copy and paste from examples that were not written with security in mind. His work commonly finds firewalls left completely open or a complete lack of authentication for components. He also warned against open sourcing the internal code a company would rely on in production, as it shows where patching is lacking and helps attackers understand the vulnerabilities they can exploit. If you do open source your code, it is vital to embrace an aggressive patching strategy.
Next, many common services, such as the foundational Interplanetary File System, IPFS, and Ceramic, a decentralized DB, present some security issues when deployed using default settings. Attackers are well aware of these flaws. The most common is that the admin interface for IPFS is always available on port 5001. Currently, there is no built-in mechanism offering port protection., meaning that unless you put your own security measures in place to guard access, anyone who can access that port can assume complete administrative control.
While every network, regardless of the technology it is using, is susceptible to attacks, Web3 systems are regularly targeted by three types of attack:
51% attacks where a group who can gain a simple majority of a network's mining resources can rewrite the whole blockchain.
Sybil attacks where an attacker creates many identities to manipulate a network.
Replay attacks where a legitimate transaction is recorded and rebroadcast in order to assume the original user's identity and steal access.
It is not all bad news; fortunately, these types of attacks are straightforward to guard against. Platforms like Cloudflare can be used to block ports and offer tunneling. A tunnel can offer DDoS protection and block all bot traffic at scale. You can also set rules to look for and block single IPs making a high number of requests.
"Unchained: Unraveling the Unconventional Security Threats in Web3" from Nick Kammerdiener
Not all vulnerabilities get a CVE
Most teams are concerned with cloud misconfigurations and the security issues they cause. There is another class of vulnerability to be aware of that might have a higher impact and is not nearly as documented: vulnerabilities within the cloud providers themselves. In his talk "Cloud vulns and keys and breaches - Oh My!" Cloud Security Engineer Jackson Reid explained that, unlike most vulnerabilities, security issues within systems like AWS, Google Cloud, and Azure do not get Common Vulnerabilities and Exposures, CVE, numbers, the accepted protocol for communicating known flaws. The worst part is there is not a lot you can do about this right now, but Jackson told us that knowing about this issue is the first step toward a more secure future.
While there is no government or official tracking system for cloud vulnerabilities, there is a dedicated group of security enthusiasts that run an open source project to document these issues, the Cloud Vulnerability DB.
He walked us through his journey of finding and responsibly reporting a bug he found on Google Cloud service keys. Once he discovered the issue, it took multiple emails and several weeks before he got a reply and resolution. Eventually, Google alerted root users of the issue and instructed everyone to rotate their service keys, but there is very little transparency about what steps they took internally or how they actually solved the issue.
He encouraged us all to engage with our cloud vendors to get better insight into issues that might impact you and your business. He ended with an example of what this can look like when done correctly, as he believes was the case with the recent Microsoft signing key issue, where there was a lot of transparency about how the issue was reported and what steps had been taken.
Cloud vulns and keys and breaches - Oh My! From Jackson Reid
Making your log data work for you
In his day-to-day work, Matt Carothers of COX Communications deals with a lot of log entries. By a lot, he means multiple petabytes worth. Every end customer they sell internet access to is allowed to hook up any device they want to their network, and each transaction produces a log entry. These are in addition to all the logs from the thousands of internal servers and processes they run. They have devised some pretty ingenious and cost-effective strategies for processing that much information at scale, some of which he shared with us In his talk "Stupid Log Tricks."
While every step of processing and storing log data has a cost, some services, such as Security Information and Event Management systems, SIEMs, are notably more expensive. Instead of pushing everything to the SIEM, they deploy a 'log broker,' a system built on Logstash and Apache Kafka to deliver logs where needed or requested. They only send about 10% of logs to the SIEM this way, and everything else is cheaply stored. They also invest in normalizing the data in terms of structure and overall timeline so that no matter where the data comes from, they are able to use a standard search process, saving a lot of human time, which is always at a premium.
A clever use of Redis
Redis is an in-memory key-value store. Unlike relational databases, there are no tables or rows. The only technical limitation to the size of a Redis deployment is RAM, which continues to get cheaper over time. Redis is at the heart of their internal tool called OIL, the observed indicator list. In OIL, every flow is put in Redis using the IP address as the key and the rest of the normalized log entry as the value. A Redis search for an IP and the relevant data takes milliseconds and gives their team an instant way to retrieve needed data. They have even built a unified web interface, allowing everyone on the security team a consistent way to search from anywhere.
Passive Asset Discovery
Most asset discovery is expensive, as the network must be mapped and continually searched for new devices. In passive asset discovery, the security logs are monitored for any new asset, and a database is built for any new entity that appears. Not only does this make it much simpler to map all assets connecting to their networks, but it also gives them immediate context on what that device is and how it behaves.
"Stupid Log Tricks" from Matt Carothers at #bsidesatl
Defending our applications together
Your author was privileged to be one of the speakers at the event as well with my session, "Who Goes There? Actively Detecting Intruders With Cyber Deception Tools." I walked everyone through the history of cyber deception and the evolution of honeytokens and then through the architecture of GitGuardian's open source ggcanary, a free way to build honeytokens using Terraform and AWS. We also took a look at some commercial options, including GitGuardian Honeytoken, which goes hand in hand with our Secrets Detection abilities, making it easier to quickly triage any breaches or code leaks.
The conversations from each track continued between the sessions and over lunch. There was a lot of fun and coffee to be had throughout the day as well. And that is what BSides is all about: bringing together folks in an otherwise fairly isolating profession. Sometimes, we do need reminders, as Sharrod gave us in her keynote, that we are all humans and all need to protect our curiosity and our peace.
I left the event feeling a part of a supporting community and I am glad for all the connections I made. Be sure to look for a BSides, or other meetup in your area. Perhaps I will see you at BSides Atlanta 2024.