Hackathons are very popular in the web3 ecosystem. Protocols and web3 products organize hackathons for several reasons: To encourage innovation, connect with developers, increase brand awareness, and more. However, the main goal is to develop products that would bring real users and value to the ecosystem. The hope is that these products and teams will eventually turn into startups that would scale. This expectation leads to significant spending on prizes and events.
In only the first quarter of 2024, DoraHacks, one of the many organizations that host hackathons for blockchain projects, has organized 24 blockchain hackathons that offer $4.4+ million in prizes exclusively online under the web3 category. In anticipation of a bull market in cryptocurrencies in 2024, we expect this number will increase rapidly throughout the year.
As Rise In, we onboard web2 developers to web3. We have been working on upskilling developers in new technologies since 2016. Based on this experience, we have always been questioning the impact of hackathons with large prizes in terms of bringing long-term success. So, this research aims to answer this question: Are web3 hackathons truly effective? Do they bring long-term success and promising products and developers to the chains? 🤔
In this study, we evaluated the effectiveness of web3 hackathons by systematically examining various factors: GitHub activity, product metrics, social media engagement, and finally, whether the participating teams continue to build on the blockchain platform that sponsored the hackathon.
Case Study 1: A blockchain protocol with little developer activity outside hackathons
In our first case study, we examined a blockchain protocol within the top 30 by market capitalization. We chose this project because of its large-prize hackathons although the protocol doesn’t have much developer education and community activities outside the hackathon. We wanted to see if large hackathons bring success by themselves.
💸This protocol held a 45-day online web3 hackathon in 2022 and distributed $430,000 in prizes. In total 102 projects were submitted. However, the top three winning projects, receiving around $300,000, don’t have active websites today. Upon reviewing their GitHub repositories, we found that two out of three projects returned a 404 error, and the third one, while accessible, had its last activity two years ago. Similarly, two out of three projects had no social media presence, and the remaining project had an inactive account.
Surprisingly, none of the winning teams showed any indication on their LinkedIn or GitHub profiles of working on the sponsoring protocol.
Out of the $130,000 awarded to the remaining 20 projects, none of the winning teams continued to develop on the sponsoring protocol despite the motivation to attract developers to their ecosystems. It was disappointing to see that none of the winning projects showed any ongoing development on their respective chains.
Given the market conditions of 2022 and the subsequent challenges faced by the crypto markets in 2023, it's essential to examine the hackathons conducted in that year. In 2023, with the impact of the bear market, the prize pool of this protocol decreased significantly from $430,000 to $50,000, distributed among five projects. But again, this time, four out of five projects had inaccessible GitHub accounts, and the accessible project had no activity for ten months. Only one of the five projects had active social media and website presence.
Case Study 2: A blockchain protocol with significant developer activity alongside blockchain hackathons
In our second case study, we focused on a project that had long been in the top ten by market capitalization and was known for regularly fostering developers with educational resources, technical support, and local communities alongside web3 hackathons. 🌿
In the 2022 hackathon, $225,000 was awarded to the top ten projects. Upon examining the GitHub activities of these projects, we found that five had received updates two years ago, four were inaccessible, and only one had ongoing GitHub activity. Regarding websites, three out of ten projects were unreachable, five were live but had yet to be updated in two years, and two were active and up to date. Regarding social media, eight out of ten projects had inactive accounts.
But the picture improved when we looked at the teams: When examining the teams behind the projects, we found that five out of ten teams were working on the sponsoring protocol, two had switched to a different network, one had returned to the web2 ecosystem, and two had no LinkedIn or GitHub activity.
Case Study 3: A platform hosting web3 hackathons of different protocols
Submissions for protocol 1: Randomly picked 12 accepted projects out of 155 projects
Winners for protocol 1: 4 runner-up projects (won $1,000 credits)
Submissions for protocol 2: Randomly picked 10 accepted projects out of 378 submitted projects
Winners for protocol 2: 8 winner projects (won $40k prizes)
In case three, we examined the results from a platform that hosts web3 hackathons of different chains. We looked into the hackathons of two different protocols, and we examined both accepted projects and winners.
Web3 Hackathon of protocol 1: Received submissions from 155 projects out of which 4 projects received prizes. We looked into 12 of the 155 submissions randomly and found that 75% of them were “spammy” submissions that joined a lot of different web3 hackathons or showed no Github or technical activity. The success rate of the 4 winning projects was better. However, one of these winners clearly advertises another competing chain on their website today instead of keeping their project on the sponsoring protocol.
Web3 Hackathon of protocol 2: Received submissions from 378 projects out of which 8 projects received prizes. However, despite only six months passing since the event, only one project has up-to-date GitHub activity among the 8 projects. Furthermore, out of 10 randomly selected projects from the submitted 378, we found that 8 of them were spamming their projects across multiple blockchain hackathons.
Our reflections on these results
As evident from the cases, web3 hackathons fail to establish a sustainable developer ecosystem despite the considerable cost. Particularly in Case 1, where the number of prizes and winning teams is significantly higher, we do not observe a successful outcome compared to our example in Case 2, with one exception.
Case 2, on the other hand, demonstrates much better results overall. Although projects may show unsuccessful outcomes over the years, we know that the protocol in question invests significantly in education outside of blockchain hackathon organizations and distributes developer rewards more widely. It also prioritizes sustainability. The most significant achievement is attributed to the majority of developers continuing to develop within the network due to these efforts. 💪🏾
In contrast to the projects examined in Case 1 and Case 2, we are not surprised by the results of Case 3 being exposed to spammy projects and consequently becoming targets for bounty hunters due to the lack of adequate education and informative meetings provided.
Overall, we suggest that if you’re a project that already has a good top-of-the-funnel developer ecosystem with local communities, developer education, and great educational resources and documentation, then web3 hackathons can be a good way to motivate the developers for the long term. But if you don’t invest in sustainable developer education and retention activities, you shouldn’t expect blockchain hackathons to create a developer ecosystem.