In the current environment of fake news, alternate facts, unverified sources, and just plain lies, it can be extremely difficult to trust what is published. However, this issue extends far beyond political news or hype stories. We find ourselves in an environment where we have more data than ever before, we rely on more data than ever before, but our ability to verify that data is stretched well beyond its limits.
The Promise and Problem with Data
The AI industry in particular has greatly increased our appetite for data. While we could always use statistical analysis on data to gain certain insights, AI has given us what seems like unlimited power, if only we have the right data to train an algorithm. Given the proper training set of data, an AI algorithm can classify different categories of data, such as identifying a product based on a photo. It can use data to predict things that would be impossible for humans to find: predicting fraudulent behavior on your bank account, how long a package will take to arrive, and whether another car is going to collide with your auto-driving vehicle. It can optimize a problem based on data, such as streamlining a factory, fitting in all your appointments based on your schedule constraints, and smoothing out a robot’s movements. With enough data an AI algorithm such as a large language model, as found in ChatGPT and similar models, can answer wide ranges of questions, often with startlingly high accuracy. In just a few short years, we have found ourselves completely dependent on AI because we are still discovering its capabilities (and limitations).
As you’ve noticed, the common denominator here is data. The problem, however, is that simply having data isn’t enough to build up a proper AI algorithm. Creating an effective AI model is actually a lot more complex and difficult than it might seem. There is a lot of expertise involved to select the right algorithm, tweak the right knobs and levers (called hyperparameters), and training/testing the model to ensure it behaves the way it should. For any of this to work, the developer must create or collect the right kind of data to use for training, label it where needed, and validate that it is correct. The amount of data needed varies but can be quite large, especially if the model is making complex decisions, choosing from many different categories, or needs to identify very small changes (like finding microscopic flaws in a product, for example).
The fact is, being able to create the right AI algorithms is becoming easier and easier, while finding (or creating) the right data to train the model is becoming more difficult as the problems we want to solve become more complex, the data required is large and complex, or we have the data but its validity is dubious. What can be done to collect and validate these trillions and trillions of data points? Let’s look at this issue and explore why blockchain’s key attributes may offer a solution, with platforms like Synesis One already showing significant promise in mobilizing a large force of people using decentralization.
The Data Collection Industry
To properly collect the data needed for an AI model, you first need to understand what problem you’re trying to solve. On one side of this industry, there are companies, organizations, and even individuals with problems to solve. In order to find or create the right data, they must be able to articulate what they are trying to solve with enough detail that data experts can understand what type of data, what type of labelling or validation, and how much of it is needed. Ideally, these entities would broadcast their problem and what type of data they need. If this were something like a report or analysis, you could use the well-established gig economy to find someone qualified for the job. The challenge is that with the data sets needed for AI, it is often a much larger job than one person can handle. However, the individual elements are usually not difficult, do not take training in most cases, and can be broken down into increments of a single data point. This is ideal for spreading out the work to many different people. If many people can each collect or create a small number of data points in a reliable way, then possibly label the data if needed using some basic guidelines, then the entity trying to solve their problem will have everything they need to get started.
Decentralization At Work
This is where blockchain, and decentralization in particular, is perfect. As mentioned above, Synesis One and other emerging Web3 firms are establishing an entire industry of data creation, collection, and validation. Blockchain is perfect because it allows smart contracts to automate the process, it allows the participants to be nearly anywhere across the globe (increasing the odds of people working the problem), and it even allows payment in the form of cryptocurrency, which is largely borderless. The result? Many different people following basic instructions to create or find the right data, validate it, perform some peer reviews if needed (to ensure everyone is doing the job right), and package it into a nice, ready-to-use data set.
While the concept might seem fairly basic, for the AI community it is, in fact, revolutionary. Data limitations have long been the curse for AI developers worldwide, with the developers often unable to build these datasets themselves, and without the resources to hire the people directly to get the job done. Only through an ultra-efficient system such as decentralization can you recruit many different people who can each contribute a small amount, be rewarded for their efforts, and all add to the pile that eventually becomes usable data.
The general population, including those working in tech, don't understand how much human labor and input is required on the data side of AI for great models to work as well as they do now.@Lempheter does an amazing job in this thread breaking down some bottlenecks in the… https://t.co/FFO3sAO3fr
— Synesis One (@synesis_one) May 2, 2024
What’s Next?
With one of the key bottlenecks of AI progress potentially solved, the decentralized data market will serve to boost the acceleration of AI in our daily lives, while also making usable and customized AI more accessible to small businesses and even individuals. We will certainly see this industry expand, if not explode, in the near future, providing a major use case in Web3 that is completely separate from DeFi, NFTs, or any of the “hype” uses that the greater public thinks of when they hear the term “blockchain.” With any luck, they might start to think of blockchain as the major boost the AI industry needs.
