Collaborative Deep Learning: Machine Learning Applications in Cryptography

In the realm of deep learning, there are instances where data from a single source is insufficient for training a model. This has led to a growing interest among data owners to not only utilize their own data but also incorporate data from other sources. One approach to facilitate this is by using a cloud-based model that can learn from multiple data sources. However, a key concern is the protection of sensitive information. 

This has given rise to the concept of collaborative deep learning, which revolves around two main strategies: sharing encrypted training data and sharing encrypted gradients. The overarching principle here is the use of fully homomorphic encryption to ensure that all data, including those used for operations in the cloud, remain encrypted throughout the learning process.

Sharing encrypted data to ensure privacy

There have been innovative approaches to ensure privacy during collaborative deep learning. One such method involves both data owners and a cloud-based system. Here’s how it works:

1. Data owners create public keys, secret keys, and evaluation keys. They then encrypt their data (like training data and desired targets) using their public keys and forward this encrypted data to the cloud.
2. The cloud, upon receiving this encrypted data, proceeds to train the model using the public and evaluation keys provided by the data owners.
3. Once the learning process updates the encrypted weights, the cloud returns these encrypted weights to the respective data owners.
4. Finally, the data owners collaboratively decrypt the received data to obtain individual updated weights. This decryption process leverages secure multi-party computation techniques.

Another more intricate method has been proposed to eliminate the need for data owners to communicate during the decryption process. This involves an additional entity, an authorized center (AU), and employs a combination of double encryption techniques and multi-key fully homomorphic encryption. The steps are:

1. Data owners create their public and secret keys and encrypt their data, which is then sent to the cloud. The AU also retains a copy of the data owners’ secret keys.
2. The cloud, after receiving the encrypted data but lacking the evaluation keys, introduces noise to the data and forwards it to the AU.
3. The AU decrypts this data using the secret keys of the data owners and re-encrypts it with a singular public key before sending it back to the cloud.
4. The cloud can now compute encrypted and updated weights using this uniformly encrypted data. Once done, the results are sent to the AU for re-encryption using individual public keys of the data owners.
5. Each data owner then receives their respective results, which they can decrypt using their secret keys.

This system has been shown to maintain semantic security, provided the public key system in use is also semantically secure. Moreover, the privacy of deep learning parameters, like weights, remains intact as long as the cloud and the AU do not conspire.

In recent advancements, there have been improvements in the basic method by introducing multi-scheme fully homomorphic encryption. This allows data owners to employ varied encryption schemes when participating in collaborative deep learning. Additionally, there have been enhancements in the accuracy of certain activation functions and an increase in the overall accuracy and speed of classification tasks compared to earlier methods.

Collaborative Deep Learning with Encrypted Gradients

An innovative approach in the realm of collaborative deep learning involves the use of additively homomorphic encryption. This method was developed as an enhancement over previous techniques that utilized asynchronous stochastic gradient descent (ASGD) as the learning method. This earlier approach was termed “gradients-selective ASGD” because it allowed each data owner to decide which gradients to share globally, ensuring their privacy. 

There was also an additional method that incorporated differential privacy by introducing Laplace noise to the gradients. Despite these measures, it was demonstrated that there was still a potential for leakage of sensitive data from the owners, even if the gradient values underwent minor modifications.

In the improved method using ASGD, the process can be outlined as:

1. Data owners retrieve the encrypted weight from the cloud, decrypting it with their secret key.
2. Using the global weight and their training data, the data owner calculates the gradient within their deep learning model.
3. This gradient, after being multiplied by the learning rate, is encrypted using the data owner’s secret key and then sent back to the cloud.
4. The cloud then updates the global weight using the encrypted data from the data owners, with the operation being limited to addition.
5. A significant highlight of this method is its robustness against potential gradient leakages. The cloud, even if it operates with a curious intent, cannot access the gradient’s information. Furthermore, when the data owner decrypts the results from the cloud, the outcome aligns perfectly with what would be expected if the cloud operations were conducted on an unencrypted gradient.

Security Implications of Machine Learning in Cryptography

The integration of machine learning into cryptography has raised several security concerns. In this section, we present a brief summary of the key findings related to this topic in recent times.

Machine Learning Security: A study from 2006 dived into the question of whether machine learning can truly be secure. This research introduced a classification of various types of attacks on machine learning systems and techniques. Furthermore, it presented defenses against these attacks and provided an analytical model illustrating the attacker’s efforts.

Expanded Taxonomy of Attacks: Building on their prior work, a subsequent study expanded the classification of attacks. This research detailed how different attack classes impact the costs for both the attacker and the defender. It also provided a comprehensive review of attacks on machine learning systems, using the statistical spam filter, SpamBayes, as a case study.

Evasion Attacks: A 2013 study introduced the concept of evasion attacks. While bearing similarities to exploratory integrity attacks, evasion attacks focus on introducing adversarial data into the training data of machine-learning-based systems. The research emphasized the importance of thoroughly assessing machine learning’s resistance to adversarial data.

Exploiting Machine Learning Classifiers: Another 2013 study highlighted a method where machine learning classifiers could be manipulated to reveal information. This research centered on the unintentional or intentional disclosure of statistical information from machine learning classifiers. A unique meta-classifier was developed, trained to hack other classifiers, and extract valuable information about their training sets. Such attacks could be used to create superior classifiers or to extract trade secrets, infringing on intellectual property rights.

Adversarial Behavior: Adversaries can potentially bypass learning approaches by altering their behavior in response to these methods. There has been limited exploration into learning techniques that can withstand attacks with guaranteed robustness. A workshop titled “Machine Learning Methods for Computer Security” was organized to foster discussions between computer security and machine learning experts. The workshop identified several research priorities, ranging from traditional machine learning applications in security to secure learning challenges and the creation of new formal methods with guaranteed security.

Beyond Traditional Computer Security: The workshop also identified potential applications beyond the conventional realm of computer security. These applications, where security concerns might arise in relation to data-driven methods, include social media spam, plagiarism detection, authorship identification, copyright enforcement, computer vision (especially biometrics), and sentiment analysis.

Security and Privacy in Machine Learning: A 2016 study provided an in-depth analysis of security and privacy concerns in machine learning. It introduced a detailed threat model for machine learning, categorizing attacks and defenses within an adversarial framework. The adversarial settings for training were divided into two main categories: those targeting privacy and those targeting integrity. Inference in adversarial settings was also categorized into white-box and black-box adversaries. The study concluded by discussing the path to achieving a robust, private, and accountable machine learning model.

Past Progress of Machine Learning in Cryptanalysis

Machine learning has been increasingly integrated into the realm of cryptanalysis, especially in enhancing the capabilities of side-channel attacks. Here’s a concise overview of its applications:

Early Incorporation of Machine Learning: One of the initial ventures into this domain involved the use of the Least Squares Support Vector Machine (LS-SVM) learning algorithm. This method targeted the software implementation of the Advanced Encryption Standard (AES) using power consumption as the side-channel. The findings highlighted the pivotal role of the machine learning algorithm’s parameters on the outcomes.

Enhancing Accuracy: A subsequent approach advocated for the use of machine learning to boost the precision of side-channel attacks. Since these attacks are based on the physical metrics of cryptosystem hardware implementations, they often rest on certain parametric assumptions. The introduction of machine learning offers a way to ease these assumptions, especially when dealing with high-dimensional feature vectors.

Neural Networks in Cryptanalysis: Another innovative method employed a neural network for cryptanalysis. This strategy trained the neural network to decrypt ciphertexts without the encryption key, leading to a notable reduction in the time and the known plaintext-ciphertext pairs required for certain encryption standards.

Expanding on Previous Work: Building on the aforementioned neural network approach, another study targeted a lightweight cipher. The focus shifted to discovering the key instead of the plaintext. The neural network’s efficiency was tested on both reduced-round and full-round versions of the cipher, tweaking network configurations to maximize accuracy.

Analyzing Encrypted Traffic: A different study delved into the analysis of encrypted network traffic on mobile devices. The goal was to discern user actions from encrypted data. By passively monitoring encrypted traffic and applying advanced machine learning techniques, they could deduce user actions with an impressive accuracy rate.

Deep Learning in Side-Channel Attacks: Deep learning was explored to refine side-channel attacks. The aim was to develop sophisticated profiling techniques to minimize assumptions in template attacks. By applying deep learning, more precise results were achieved in side-channel attacks on certain encryption standards.

Counteracting Machine Learning Attacks: A unique approach was introduced to thwart machine learning from being weaponized against Physical Unclonable Functions (PUFs) in lightweight authentication. This method combined a lightweight PUF-based authentication with a lockdown technique, ensuring machine learning couldn’t successfully extract the new challenge-response pair.

Conclusion

The integration of machine learning into cryptography has opened up new avenues for enhancing security and optimizing processes. While it offers promising solutions, especially in collaborative deep learning and cryptanalysis, there are inherent security concerns that need addressing. As the field evolves, it’s crucial for researchers and practitioners to be aware of potential vulnerabilities and work towards creating robust, secure systems.