Development of a Simultaneous Quantification for Hydrogen and Ammonia Using Deep Neural Network and The Rotating Ring Disk Electrode Method

Abstract

Ammonia is considered as a viable candidate for a hydrogen carrier in a hydrogen-based society. Currently, real-time analysis of products in the development of highly active ammonia electrolytic synthesis catalysts is limited to the extremely expensive technique of electrochemical mass spectrometry. In this study, we aimed to develop a real-time simultaneous quantification method for hydrogen and ammonia concentrations using the rotating ring-disk electrode technique. We adopted deep neural network-based machine learning technology using the data from cyclic voltammogram (CV) measurements on a Pt ring electrode. The sum of squared residuals decreased to 1/0.645 at hydrogen partial pressure and 1/92.8 at ammonia concentration compared to the conventional combination of nonlinear least-squares method and solving simultaneous equations. Moreover, classifying concentrations through image recognition based on the obtained CV images resulted in successful concentration determination with an accuracy of 78.9%.

1 INTRODUCTION

Machine learning is a technique that involves learning from data to predict unknown information. In the field of electrochemistry, there have been reports on various applications, including the exploration of battery materials [1], classification of reaction mechanisms [2], and qualitative and quantitative analysis of explosives [3].

Ammonia is one of the most synthesized chemicals globally, and widely used as a raw material for many products. Additionally, ammonia is gaining attention as a promising fuel and energy carrier in a hydrogen-based society due to its high hydrogen density and ease of liquefaction [4].

The bulk of current ammonia production depends on the Haber-Bosch process. This process requires high-temperature and high-pressure conditions (400–600 °C, 20–40 MPa) [5], necessitating stabilization equipment, such as batteries, for on-site ammonia synthesis using intermittent renewable energy. The electrochemical synthesis of ammonia under low temperatures and atmospheric pressure is proposed as an ideal synthesis method for the on-site utilization of renewable energy. However, electrolytic nitrogen reduction is impeded by low selectivity, limited nitrogen adsorption, and slow production rates [6], posing challenges to its practical applications.

Three mechanisms were proposed for the nitrogen reduction reaction (NRR): dissociative pathway, associative pathway, and enzymatic pathway [7]. In the dissociative pathway, a nitrogen molecule adsorbs onto the catalyst surface, followed by the triple bond is broken before protonation occurs. This pathway is limited by the strong triple bond between nitrogen atoms. Whereas for the associative pathway, two nitrogen atoms adsorbed on the catalyst surface keep their bond during protonation. The enzymatic pathway involves hydrogenation proceeding with a nitrogen molecule binding via a side-on configuration. On the other hand, the catalyst surface can also serve as a site for hydrogen atom adsorption and proton reduction promotion.

In the development of highly active electrochemical nitrogen reduction catalysts, hydrogen generation occurs concurrently with nitrogen reduction due to the competing proton reduction reaction [8]. Thus, quantifying the proportions of these reactions is crucial. The detection of products requires using costly electrochemical mass spectrometry or conducting analyses of both the electrolyte and a gas-phase. In the latter approach, a two-compartment cell is used to prevent the consumption of the products at the counter electrode, requiring the quantification of hydrogen gas through gas chromatography and ammonium ion through ion chromatography or the indophenol method [9].

The rotating ring-disk electrode (RRDE) method is an inexpensive analytical technique. This technique enables the quantification of products formed on the disk electrode via the ring electrode and is widely applied in the analysis of oxygen reduction reactions. In evaluating the performance of oxygen reduction catalysts, the amount of hydrogen peroxide produced is typically estimated by monitoring the ring-current while maintaining the ring potential constant at 1.4 V vs. RHE, where hydrogen peroxide can be oxidized. However, this method provides insufficient information for quantifying multiple materials where the relationship between current and concentration is nonlinear. Ring cyclic voltammetry (CV), on the other hand, offers current information at multiple potentials, thereby enabling the acquisition of more detailed information. However, interpreting CVs is particularly complex, as their profiles are influenced not only by the redox reactions of products but also by electrode surface reactions and adsorbed species on the electrode.

In this research, our objective was to simultaneously quantify ammonia and hydrogen by applying machine learning techniques, including deep neural networks and image recognition based on the continuous CV measurements on ring electrode (Figure 1).

Figure 1.

 Illustration of ammonia and hydrogen simultaneous determination by a RRDE method.

2 EXPERIMENTAL

2.1 Chemicals

Potassium dihydrogen phosphate (Wako, pH standard grade), dipotassium hydrogen phosphate (Hayashi Pure Chemical, special grade), ammonia solution (Kishida chemical, for hazardous metal analysis), ammonium hexafluoro silicate (Kanto Chemical), argon gas (Yokohama Chemical), and hydrogen gas (Resonac Gas Products) were employed in this study.

2.2 Electrochemical measurements

A Pt ring-Pt disk electrode (ring inner diameter 7 mm, outer diameter 9 mm, electrode area 25.13 mm²) was employed as the working electrode, an Ag|AgCl electrode served as the reference electrode, and a Pt electrode was used as the counter electrode (Figure 2). The RRDE measurements were performed using a potentiostat (PS-08, Toho-Giken), a function generator (HB-105, HokuTo-Denko), and a motor speed controller (SC-05, Nikkou Keisoku) in 100 mM K₂HPO₄ − 20 mM KH₂PO₄ phosphate buffer solution (pH 7). All potentials are quoted with respect to the reversible hydrogen electrode (RHE).

Figure 2.

 Illustration of an electrochemical measurement configuration.

To measure the response of the Pt ring electrode under known concentrations of hydrogen and ammonium ions, the system was purged for 20 minutes with 100 mL/min H₂-Ar gas at any desired partial pressure. Subsequently, the disk electrode was open-circuited, and CV measurements of the Pt ring electrode (1.4 to 0.15 V, 600 rpm) were conducted with varying sweep rates of 500 and 250 mV/s. The measurements were performed by altering the ammonium ion concentration (10⁻⁷ to 10⁻¹ M) and hydrogen partial pressure (1 to 0 atm). Prior to each measurement, a potential of 1.5 Vwas applied to the Pt ring for 1 minute to perform cleaning of the ring electrode surface.

2.3 Data analysis

The obtained 50 CVs were linearly interpolated per cycle, and the data points were fixed at 400 for each revolution. The resulting CVs underwent smoothing using scipy Savitzky-Golay filter of python language and were standardized for ease of machine learning. The standardized CVs using the DNN analysis were calculated using equation 1 as follows:

  

$\begin{array}{c}{i}_{\text{CV}\_\text{std},j}=\frac{i_{\text{CV},j}-{\overline{i}}_{\text{CV}}}{\sqrt{\frac{1}{n}{\sum }_{k=1}^n{\left(i_{\text{CV},k}-{\overline{i}}_{\text{CV}}\right)}^2}}\end{array}$(1)

Where is the average of whole current in 1 cycle, i_CV,k is each current value, and n is the number of current measured in 1 cycle of CV.

PyTorch was employed as a machine learning tool to utilize deep neural networks (DNNs). DNNs are one of the most popular machine learning tools for analyzing hidden nonlinear relationships [10]. DNNs consist of an input layer, an output layer, and intermediate hidden layers. In this study, the input layer consisted of CVs data, and the output layer consisted of ammonium ion concentration and hydrogen partial pressure. Nodes in each layer were interconnected, forming associations that varied in weight through linear combinations and activation functions. The differences between the obtained values at the output layer and actual answers were evaluated by the loss function. The disparity between the output values and the expected results was assessed using a mean squared error loss (MSELoss) function. Furthermore, the DNN parameters were optimized using the RMSprop algorithm. The activation function, optimization function, and loss function were selected by updating the parameters while keeping other parameters fixed, and choosing the ones with the highest accuracy. Finally, we used the ReLU function as the activation function in our data estimation model.

To assess potential overfitting, we performed K-fold cross-validation in estimating ammonia concentration. The data set was divided into five subsets, with one subset serving as validation data and the remaining four as training data. We evaluated the presence of overfitting by conducting five rounds of learning and analyzing the performance of each iteration.

For comparative analysis, we employed the conventional nonlinear least-squares method to derive an approximation formula, which was then used to compute the concentrations of hydrogen and ammonia from a set of simultaneous equations. We calculated the parameters of the following equation 2 for the relationship between hydrogen partial pressure (x axis) − ammonia concentration (y axis) and current value (z axis) using scipy curve_fit function of python language:

  

$\begin{array}{c}{i}_E=a_0+a_{1}H+a_{2}N+a_3{H}^2+a_{4}HN+a_5{N}^2+\\ a_6{H}^3+a_7{H}^{2}N+a_{8}H{N}^2+a_9{N}^3\end{array}$(2)

where i_E is a ring-current, N is log₁₀[NH₃], and H is the partial pressure of H₂. We calculated the concentrations of ammonia and hydrogen by solving the system of equations obtained by substituting independently measured ring-current values at two different potentials (0.2 V and 0.8 V), around which either hydrogen oxidation or a combination of hydrogen oxidation and ammonia oxidation occurs, into equation 2, using the nleqslv function (Broyden’s method) in R language. Current at 0.2 and 0.8 V were used for the conventional nonlinear least square method.

2.4 Estimation of concentration with image classification technology

We visualized the obtained CVs at a resolution of 216 ppi, and utilized the MobileNetV2 [11] architecture as a pre-trained model for transfer learning architecture as a pre-trained model for transfer learning for image-based recognition to classify the concentrations of ammonia and hydrogen partial pressure. Due to the limited number of images for training, data augmentation technique was applied, introducing 3% noise to the original data set, thereby expanding the data set to 4000 sample. It is important to note that the technique used in this study is not data augmentation in the context of deep learning. Instead, due to the scarcity of experimental data, we prepared our data set by artificially introducing random noise into the actual data. For the purposes of this experiment, we will assume that this data set was derived from real experimental data. For the purposes of this experiment, we will assume that this data set was derived from real experimental data. In our optimization algorithm, we employed Adam (Adaptive Moment Estimation). The loss function was defined as sparse categorical cross-entropy. This combination was selected based on its superior accuracy compared to other configurations tested. To mitigate overfitting, we implemented cross-validation, allocating 20% of the data set for validation purposes. The model’s performance was evaluated using accuracy as the primary metric.

3 RESULTS AND DISCUSSION

3.1 Ring CV and electrode reaction

The CVs of the Pt ring electrode at various hydrogen partial pressures and ammonia concentrations are shown in Figure 3. As the hydrogen partial pressure increases, the anodic current originating from hydrogen oxidation also increases (Figure 3 (B)). In region (i), during the positive sweep, hydrogen desorption occurs, while during the negative sweep, hydrogen sorption takes place [12]. In the positive sweep of region (ii), both the oxidation reaction of ammonia and the formation of the Pt oxide occurs, whereas in the negative sweep, the reduction of the platinum oxide takes place [13].

Figure 3.

 CVs of Pt ring electrode in 0.1 M NaH₂PO₄-0.02 M K₂HPO₄ phosphate buffer (pH 7) (a) without or (b) with 0.1 M NH₃ at various partial pressures of H₂. Scan rate was 250 mV s⁻¹.

3.2 Estimation using Conventional Nonlinear least-squares method

The parameters obtained from the analysis using the conventional nonlinear least-squares method are summarized in Table 1. The coefficients of determination were high values, 0.994 and 0.992 at 0.2 V and 0.8 V, respectively. Figure 4 shows the relationship between hydrogen partial pressure, ammonia concentration, and current value at 0.2 V and 0.8 V. The surfaces plotted in this Figure are based on the parameters obtained from the analysis, according to equation 2. The sensitivity to ammonia concentration was significantly lower compared to that for hydrogen partial pressure.

Table 1. Parameters (0.2, 0.8 V) of eq. 2 obtained by a nonlinear-least square method.

×103	0.2 V	0.8 V
a 0	48.7 ± 38.2	−376 ± 37.0
a 1	661 ± 35.7	323 ± 34.6
a 2	97.9 ± 41.0	146 ± 39.7
a 3	14.6 ± 22.8	−317 ± 22.1
a 4	−24.1 ± 23.0	−44.2 ± 22.3
a 5	7.31 ± 4.42	−68.4 ± 4.28
a 6	13.6 ± 10.8	−54.5 ± 10.5
a 7	0.656 ± 0.278	−4.45 ± 0.269
a 8	3.94 ± 0.950	−2.42 ± 0.921
a 9	28.8 ± 4.06	16.7 ± 3.94

Figure 4.

 Dependence of experimental and predicted anodic current at (A) 0.2 V or (B) 0.8 V on ammonia concentration and partial pressure of H₂. Scan rate is 250 mV s⁻¹.

The predicted values for ammonia concentration and hydrogen partial pressure were obtained using equation 2 with independently measured data. In this study, the coefficients of determination for the logarithm of ammonium ion concentration (R_N²) and the partial pressure of hydrogen (R_H²) was used as an indicator of prediction accuracy, and are expressed as follows:

  

$\begin{array}{c}{R}_{\text{N}}^2=1-\frac{\sum {\left(N_{\text{exp}}-N_{\text{p}}\right)}^2}{\sum {\left(N_{\text{exp}}-{\overline{N}}_{\text{exp}}\right)}^2}\end{array}$(3)

  

$\begin{array}{c}{R}_{\text{H}}^2=1-\frac{\sum {\left(H_{\text{exp}}-H_{\text{p}}\right)}^2}{\sum {\left(H_{\text{exp}}-{\overline{H}}_{\text{exp}}\right)}^2}\end{array}$(4)

where N_exp and H_exp are the experimental values of the logarithm of ammonium ion concentration and hydrogen partial pressure, respectively, N_p and H_p are the predicted values of the logarithm of ammonium ion concentration and hydrogen partial pressure, respectively. and are the average of the experimental values.

R_H² and R_N² were estimated as 0.949 and −2.793, respectively. Hydrogen partial pressure could be predicted despite certain discrepancies in areas of high concentration, while predictability for ammonia concentration was limited to levels exceeding 1 mM, as shown in Figure 5. Simultaneous quantification of hydrogen and ammonia was not possible with the conventional nonlinear least-squares method in this system.

Figure 5.

 Plots of predicted ammonia concentration (N_p) by Broyden’s method vs. experimental value.

3.3 DNN method and its verification (DNN)

Because the optimal number of nodes for DNN is unknown, we employed “optuna,” that can optimize functions for python language, to search for the nodes that minimize |1−R_N²| and |1−R_H²|. To provide an integrative evaluation of two parameters, we introduced a parameter d², associated with the squared Euclidean distance, defined by the following equation:

  

$\begin{array}{c}{d}_k^2={\left(\frac{\left|1-R_{\text{N},k}^2\right|-|1-R_{\text{N}}^2{|}_{\text{min}}}{|1-R_{\text{N}}^2{|}_{\text{max}}-|1-R_{\text{N}}^2{|}_{\text{min}}}\right)}^2\\ +{\left(\frac{\left|1-R_{\text{H},k}^2\right|-|1-R_{\text{H}}^2{|}_{\text{min}}}{|1-R_{\text{H}}^2{|}_{\text{max}}-|1-R_{\text{H}}^2{|}_{\text{min}}}\right)}^2\end{array}$(5)

where, |1−R_N²|_max and |1−R_H²|_max are the maximum values of |1−R_N,k²| and |1−R_H,k²| excluding outliers, and |1−R_N²|_min and |1−R_H²|_min are the minimum values of |1−R_N,k²| and |1−R_H,k²|, excluding outliers, respectively. The outlier points were estimated by the box plot rule.

Tables 2 to 4 present the top five d² explored by optuna for configurations with 2, 3, and 4 hidden layers, respectively. The results indicate that the configurations of 22, 33, and 11 nodes in the hidden layers were the closest proximity to the origin in the normalized space of |1−R_N²| and |1−R_H²|. In subsequent models in this research, we employed the configuration with nodes 22, 33, and 11 in the hidden layers, respectively.

Table 2. Top of d² nodes explored by optuna. (hidden layer: 2)

rank	hidden layer node num.		|1−RN2|	|1−RH2|	d 2
	1	2
1	32	11	0.0398	0.0420	0.0584
2	23	25	0.0401	0.0422	0.0609
3	13	63	0.0421	0.0413	0.0667
4	32	14	0.0430	0.0402	0.0674
5	18	11	0.0442	0.0392	0.0727

Table 3. Top of d² nodes explored by optuna. (hidden layer: 3)

rank	hidden layer node num.			|1−RN2|	|1−RH2|	d 2
	1	2	3
1	22	33	11	0.0423	0.0377	0.0477
2	65	20	11	0.0427	0.0388	0.0564
3	51	13	41	0.0427	0.0389	0.0580
4	58	14	20	0.0419	0.0429	0.0772
5	58	13	55	0.0414	0.0449	0.0895

Table 4. Top of d² nodes explored by optuna. (hidden layer: 4)

rank	hidden layer node num.				|1−RN2|	|1−RH2|	d 2
	1	2	3	4
1	90	66	15	70	0.0428	0.0423	0.0803
2	58	23	42	94	0.0429	0.0438	0.0915
3	94	20	46	81	0.0467	0.0415	0.116
4	37	19	46	81	0.0478	0.0393	0.118
5	17	10	42	15	0.0416	0.0482	0.122

R_N² and R_H² were found to be 0.959 and 0.921, respectively (Figure 6). While R_N² significantly improved compared to Broyden, R_H² exhibited a slight decrease in accuracy around 1 atom. This is believed to be due to the standardization in equation 1, which cuts off a substantial portion of the information related to hydrogen oxidation current. Thus, we performed normalization using equation 6, which does not reduce information about the hydrogen oxidation current as follows:

  

$\begin{array}{c}{i}_{\text{std},E}=\frac{i_E-{\overline{i}}_E}{\sqrt{\frac{1}{n}{\sum }_{k=1}^n{\left(i_{E,k}-{\overline{i}}_E\right)}^2}}\end{array}$(6)

where _E is the average current at corresponding time ( = same potential) in all data sets, i_E is each current value, and n is the number of measured in data sets. Adopting equation 6 improved the prediction accuracy of hydrogen partial pressure (R_H² = 0.995) compared to equation 1, while it led to a decrease in accuracy for ammonia concentration (R_N² = 0.877).

Figure 6.

 Comparison of each optimization in (A) ammonia concentration and (B) hydrogen partial pressure. The symbols are the average values predicted for 17 experimental points per condition.

Figure 7 shows the learning curve in K-fold cross-validation. The loss function converged to similar values across all folds. Additionally, the relative error of R_N² for the validation data was 0.481%, with little variation. This suggests that the effect of overfitting was suppressed in this learning process.

Figure 7.

 Learning curves of machine learning using K-Fold cross-validation.

3.4 Density prediction by image recognition

Figure 8 illustrates the results of classifying CV images into 40 categories based on hydrogen partial pressure and ammonia concentration using image recognition. The concurrent increase in both epoch count, and test data accuracy indicates effective mitigation of overfitting. Notably, the classification accuracy on the test data reached a high precision of 78.9%. These results demonstrate that despite the addition of 3% noise to the CV data, the model maintained approximately 80% prediction accuracy, suggesting robustness against noise. In contrast, the DNN (equations 1 and 6) did not incorporate artificial noise, potentially rendering it more susceptible to measurement errors. However, it is important to note that the current method is classification-based. For more detailed concentration predictions, the development of a regression model with comparable accuracy is anticipated. The model achieves a high level of accuracy for both training and validation, making it a valuable tool for image classification tasks involving 40 different classes. Transfer learning has evidently contributed positively, allowing the model to leverage pre-trained features, resulting in faster convergence and higher accuracy. Future research can further enhance the model’s performance by implementing additional data augmentation techniques to create more diverse training samples, which can help in reducing overfitting and improving validation accuracy. Adding regularization techniques such as dropout or L2 regularization can also prevent overfitting. Implementing a learning rate scheduler to dynamically adjust the learning rate during training can assist in fine-tuning the model.

Figure 8.

 Density classification using image recognition.

Experimenting with different model architectures or deeper networks can capture more complex patterns in the data. These improvements are expected to strengthen the model’s generalization ability and enhance overall performance.

4 CONCLUSION

In this research, we tackled the simultaneous quantification of ammonia concentration and hydrogen partial pressure using machine learning techniques applied to CV data derived from the ring electrode of a rotating ring-disk electrode. The approach employing DNNs significantly enhanced accuracy compared to the conventional method of concentration determination, which combines nonlinear least-squares for parameter determination with simultaneous equations. Additionally, it was found that specific data normalization methods are more effective in improving the accuracy of concentration determination for each substance, even when the same data set is used. Furthermore, image recognition using a pre-trained model achieved a classification accuracy of 78.9%. The study demonstrated that using machine learning with DNNs enables quantification of concentrations from nonlinear data, which is difficult to discern by humans.

REFERENCES

• [1]   H. S. Stein, A. Sanin, F. Rahmanian, B. Zhang, M. Vogler, J. K. Flowers, et al., Curr. Opin. Electrochem., 35, 101053 (2022). doi:10.1016/j.coelec.2022.101053
• [2]   A. M. Bond, J. Zhang, L. Gundry, G. F. Kennedy, Curr. Opin. Electrochem., 34, 101009 (2022). doi:10.1016/j.coelec.2022.101009
• [3]   X. Cetó, A. M. O’ Mahony, J. Wang, M. del Valle, Talanta, 107, 270 (2013). , doi:10.1016/j.talanta.2012.12.042 PMID:23598222
• [4]   D. A. Chisalita, L. Petrescu, C.-C. Cormos, Renew. Sustain. Energy Rev., 130, 109964 (2020). doi:10.1016/j.rser.2020.109964
• [5]   M. Kitano, Y. Inoue, Y. Yamazaki, F. Hayashi, S. Kanbara, S. Matsuishi, et al., Nat. Chem., 4, 934 (2012). , doi:10.1038/nchem.1476 PMID:23089869
• [6]   M. Arif, M. Babar, U. Azhar, M. Sagir, M. Bilal Tahir, M. Asim Mushtaq, et al., Chem. Eng. J., 451, 138320 (2023). doi:10.1016/j.cej.2022.138320
• [7]   F. Rehman, M. Delowar Hossain, A. Tyagi, D. Lu, B. Yuan, Z. Luo, Mater. Today, 44, 136 (2021). doi:10.1016/j.mattod.2020.09.006
• [8]   S. Assad, T. Tariq, M. Zaeem Idrees, A. Mannan Butt, K. Bakhat, U. Shamraiz, J. Electroanal. Chem. (Lausanne), 931, 117174 (2023). doi:10.1016/j.jelechem.2023.117174
• [9]   R. Konno, H. Shiroishi, et al., IEICE Tech, 121, 5 (2022).
• [10]   P. Puthongkham, S. Wirojsaengthong, A. Suea-Ngam, Analyst, 146, 6351 (2021). , doi:10.1039/D1AN01148K PMID:34585185
• [11]   M. Sandler, A. Howord, et al., MobileNetV2: Inverted Residuals and Linear Bottlenecks, https://arxiv.org/pdf/1801.04381.pdf, 23/05/06. doi:10.1109/CVPR.2018.00474/arXiv.1801.04381
• [12]   T. Ioroi, K. Yasuda, Electrochemistry (Tokyo), 77, 263 (2009). doi:10.5796/electrochemistry.77.263
• [13]   T. Lopes, J. Chlistunoff, J.-M. Sansiñena, F. H. Garzon, Int. J. Hydrogen Energy, 37, 5202 (2012). doi:10.1016/j.ijhydene.2011.12.040