Botting programs that swarmed NFT mints were responsible for Solana’s April 30 network crash.
Metaplex, a NFT protocol manufacturer, has instituted a “bottax” and is currently developing additional measures to combat such issues. Solana is also reworking its network fees.
Amid a rising market for Solana NFT collections–headlined by recently hot projects like Okay Bears and DeGods–has come an increase in malicious programs used to unfairly game new NFT launches on the blockchain. It crashed the entire Solana Network on April 30th.
According to Solana Labs’ postmortem report, these programs (or bots), sent an automated barrage transaction on Saturday that overwhelmed the blockchain network. It was a total of 6 million transactions per second. They did this to try and beat legitimate users during the minting process in which collectors buy newly generated FFT collectibles.
The bots attacked Candy Machine, the minting device for Solana NFT protocol Metaplex and shut down the entire network. Solana was rendered unusable. This caused a scramble among validators and contributors to fix the problem and bring the network online. It took seven hours for Solana to be restored to normal.
Over the past year, platforms such as Solana or Ethereum have been captivated by the rise of NFTs, unique tokens that can be used to prove ownership over digital assets like collectibles. Solana is quickly gaining popularity, despite Ethereum being the most popular, with billions in NFT trading volume every month.
According to data from DappRadar in April, Solana NFTs collectively generated $295 million in sales. This is a 91% jump month-over-month. According to CryptoSlam Solana has witnessed over $2.2 billion in NFT trading volume so far. The rising market is driving significant developer and user activity to the platform.
Solana’s popularity in the NFT space has been partly due to its fast transactions and low fees, which are real differentiators from Ethereum. Nhan Phan, Metaplex Studios’ CTO, said that the network is “kind of open to bots”.
Solana’s key features that make it attractive for NFT buyers are also making the platform vulnerable to attack. This is not the first time Solana’s network has been under immense strain. Last September’s downtime was caused by a token launch at DeFi protocol. It lasted more than 17 hours. It’s not the first time that NFT-related activity has been pinned.
Metaplex and Solana Labs began to share plans after the crash. They hoped to resolve the NFT botting issue and prevent it from happening again. It all starts with a “bottax.”
Let the bots pay
Metaplex’s botting penal was introduced shortly after Solana returned online. It charges these programs a fee or tax for submitting “invalid transactions”–that is, large numbers of transactions that have failed that were identified as coming from an automated program that “blindly attempts to mint,” per tweets by the company.
Before its implementation, there were no real disadvantages to using botting programs for flooding Candy Machine mints with new NFTs and removing legitimate users. A bot could send out 100,000 transactions in an NFT mint and then successfully minte 100 NFTs using the completed transactions. There would be no penalty for the remaining 99,900.
Phan said that they were trying to smash the Candy Machine program right as the mint opened, so no users could get in.” Phan said that they then would leave their bots on for ever, which was annoying for everyone. They were just like “OK, whatever it doesn’t matter” because there wasn’t any real monetary impact.
There is a 0.01 SOL Tax on transactions identified as coming from bots. While that’s only a $0.50 individual fee, it could add up to a lot for anyone trying to overthrow NFT mints on Solana.
The tax generated 1,620 SOL (nearly 82,000 in penalties) by Wednesday. These funds are paid to the creators each NFT mint that was affected. Phan said that creators have been paid by bots. This is an interesting and humorous thing.
The Metaplex protocol updates make the Solana network even more efficient. Phan stated that the update allows validators to quickly process and validate these transactions rather than having to go through the “churn of trying agree to whether it’s invalid or not.” This reduces the burden on the wider network.
Phan claims that malicious botting in NFT mins on Solana has “go down by an order-of-magnitude” since the Metaplex upgrade. However, the SOL count above indicates that there are still some botters trying to cheat the system with 162,000 NFT botting transactions.
Conflict amid chaos
In the chaos of bringing Solana back online, April 30th and May 1st, network validators provided restart instructions which included the option (but not a requirement) to temporarily block the Candy Machine program in order to avoid any botting problems during restart.
There was also talk among validators about blocking other NFT-related smart contract–the code which powers NFTs, decentralized applications, and NFTs–at that time, this included the Magic Eden, a leading Solana marketplace. Jack Lu, the founder of the marketplace, stated in a Discord message to validators that “Pls DO not BLOCK THIS ADDRESS PLS.”
Lu stated that they were informed by validators on Saturday (April 30) that they were considering closing down our contract and keeping other marketplace contracts open. “This was not a statement of Web3 values to us.”
He continued, “We also discovered that the methodology used to make these decisions was not clear. Therefore, we were forced to act immediately to try and protect our marketplace from being shut-down selectively and at exclusion of all other marketplaces.” “Fortunately, our marketplace contract wasn’t blocked and the network was restored.”
The idea that Solana validators were asked to consider blocking NFT related apps and services was met with significant backlash from social media. Tweets were critical of perceived censorship within the Solana network. Although intended to be temporary, the idea that transactions could be censored at layer-1 could be seen as a slippery slope.
These instructions were allegedly written by and compiled in the community by validators, but they were widely shared by the Solana Foundation founders and the network.
Hudson Jameson, , was formerly of The Ethereum Foundation . He tweeted the endorsement of optionally compiled instructions to be censored. He also added that this “sets precedents about censorship at L1.”
Austin Federa, Solana Labs’ head of communications, has rebuffed many of these tweets personally. In one instance, Federa tweeted that Lu was “taken completely from context” and that anyone could propose any reason.
Phan described the instructions of the validators as “temporary, optionally blocking” to aid in getting the network back up and running.
He said, “It was absolutely our intention to unblock all,” and noted that Candy Machine was the focus of the instructions. “This was primarily a collaboration between the validators and us in the interests of the ecosystem.”
It’s an arms-race
The “bot tax” is now live, and the Solana network has not experienced significant problems since it returned online on May 1. It seems that the NFT mint botting problem has been resolved–at least for the moment.
Phan and Metaplex don’t think it will remain that way forever. He called the battle for botters and protocol developers “an everlasting arms race” and said that those seeking to mint potentially valuable NFTs can continue to search for ways to get around it.
He said, “As long there are economic incentives […] for people to do botting then there will be, right?” “I think that the beauty of decentralized, censorship resistant systems is that anybody can do this stuff. This is also one of the downsides.”
He described the botting penalty “step one, out of many steps” to protect NFT mins and wider Solana network, but didn’t elaborate on potential next moves. “If I shared these, then the bots could know.”
Solana may provide ongoing relief by reworking their fee model to allow users to bid for priority when submitting transactions. Solana’s postmortem from the April 30 crash stated that “fees were coming to Solana”, but it’s actually “fee priority” that’s happening. Once changes are made, spamming the network could be very costly.
Metaplex won’t let the Candy Machine-led network fail to keep it going. The team is currently working on a next generation NFT specification. This will include performance improvements and cost reductions. It will also have modularity features.
Phan stated that the new specification will allow creators to pursue all of the amazing, crazy, and new things people want to do with NFTs.
