Automated Cheque Processing Through Data Verification and Siamese Networks Anil Muthigi 1 , Ashutosh Kumar 1 , Gaurav Bhagchandani 1 , Vijay Nath 2 anilmuthigi123@gmail.com , ashutoshkumarranchi1@gmail.com , gauravbhagchandani51@gmai l.com , vijaynath@bitmesra.ac.in 1. Department of CSE, B.I.T. Mesra, Ranchi - 835215(JH), India 2. Department of ECE, B.I.T. Mesra, Ranchi - 835215(JH), India Abstract - Bank Cheques are still majorly used for financial transactions all over the world. A large number of che ques are processed manually on a daily basis, thereby requiring a lot of time, money and human effort. In such a manual verification, info rmation like date, s ignature, amount present on the cheque has to be physically verified. This paper aims at finding a solution for processing cheques which increases the efficiency of this process, while minimizing human intervention The service was hosted locally o n a webpage , we first accepted t he cheque image from the user, passed it to Open CV which returned the various parts of the cheque which were the n passed to G oogle V ision API to convert them into text , but MICR code was passed to tesseract OCR. After the successful extraction, the details were verified from the information in the SQLite database and signature was verified using the model trained in Jupyter Notebook. 1. Introduction Due to security and trust issues, paper cheques are stil l estima ted to play a big role in financial transactions worldwide. For clearance, the cheque is first converted into its digital form and then passed onto the cheque clearing unit for further processing of the cheque, which involves visual verification of all the details and digital transfer of cheque details between banks for amount confirmation and for validating the amount transfer. 2. Extracting cheque details Different bank cheques have different size, shape and different relative posit ion of the fields in a cheque. A different searching region bounded by coordinates can be determined for every field in the cheque which can then be extracted from their regions after being grayscale d 2.1 Noise reduction Different cheques have a diff erent background styles which thereby bring certain unwanted errors (“noise”) in the extraction of text from the different fields in the cheque. To avoid extracting unnecessary details we can convert the extracted text into greyscale format and use other t echniques like erosion - dilation to enhance the quality of extracted information. 2.2 Extracting fields from their regions As demonstrated in Fig - 1 the cheque image can be broken down into different bounding boxes based on their relative positions in the cheque to extract various fields from the cheque image. This process is called image slicing. 2.3 Extracting MICR code For Indian banks, there is a common pattern of the location of MICR (Magnetic Ink Character Recognition) code in the bank cheque. The MICR code is always found to be present in the lower 10% of the cheque image. The MICR code is written using a special font which can only be extracted from an optical character recognition model specifically trained to read that font. Example of one such optical character recognition model is the Tesseract OCR. The extracted MICR code is divided into 4 parts. The first part of the code is the cheque number. The second part of the code is divided into three parts - the first part denotes the cit y name; the second part denotes the bank code to distinguish between different banks in India and the third part is the branch code which distinguishes different branches of the same bank. The third part of the code is the RBI ( Reserve bank of India) code and the fourth part denotes the transaction number. Thus, we can see that extracting the MICR code will actually help us uniquely identify the payer’s bank details. 3. Data Verification After extracting the data, we firstly need to verify the extracted cheque details for cheque validity. The payee’s name field on a cheque denotes the name of the person to whom the money is to be paid through that cheque. The payee - name field on a cheque is used to write the name of the person to whom the money is paid through that cheque. 3.1 Verifying cheque details In India, payee name is located above the amount field as shown in Fig - 1. For a cheque to be processed without any errors the payee’s name should match with the given account number, the account number on the cheque should match with that of the payer, the signature of the payer should match with that given in the database and the payer should have enough amount in his/her balance in or der to complete the transaction. In India, cheques cannot be processed after 3 months of the date written on the cheque, hence verifying the date of when the cheque was issued also has its importance. 3.2 Signature Verification This is not only the most i mportant step in the cheque processing industry but also the most sensitive out of all the cheque details. The thing that makes it so sensitive is that there is no exact measure to quantify if a signature is genuine or not. If the given signature seems to be similar to the one given in the database it is considered to be genuine. Hence, it is obvious that there will be certain human bias involved in what is considered to be genuine and what is not, thus making this a very non - uniform process. 3.2.1 Siamese Networks As shown in Fig - 2, Siamese Networks (also called as twin - networks) is a special type of CNN (Convolutional Neural Network) model in which two or more inputs are encoded into vector embeddings and their distance is computed. The contrastive loss function takes in the output of this network and treats them as vectors pointing in a multi - dimensional vector space. The loss function computes its loss in a way so as to minimize the distance between similar (positive) samples and maximize the distance between dissimilar (negative) samples. 3.2.2 Contrastive loss The mathematical expression of the contrastive loss function is as follows: Contrastive loss is a Distance - based Los s function ( which can be either cosine distance or Euclidian Fig - 1: Extracting cheque details from an Indian Bank cheque distance). It tries to ensure that semantically similar examples are embedded close together. The original sample has a margin(m) around its vector space. This loss function is mathematically def ined to push the negative samples outside of the neighborhood by a margin while keeping positive samples within the neighborhood. 4. Comparison with Current state of the art models on CEDAR Dataset State of the art models #Signatu res Accura cy FAR FRR Word Shape 55 78.50 19.5 0 22.4 5 Graph Matching 55 92.10 8.20 7.70 Zernike Moments 55 83.60 16.3 0 16.6 0 Surrounde d ness features 55 91.67 8.33 8.33 Signet Model (Convolutio nal Siamese Networks 55 86.5 13.7 0 13.1 5 5 . Conclusion When we are dealing with issues as sensitive as money, experimentations and risks are not very much appreciated. An automated cheque processing system can therefore be introduced into the process only if it offers reliability. With advanced technologies an d further research, accuracy of the model can be further enhanced and the cheque clearance process can be truly automated and put to industrial use . There is no denying the fact that achieving human - level accuracy and understanding is yet to be achieved, but it is definitely a step in the right direction. 6 . References 1. Bromley, Jane, Bentz, James W, Bottou, Le ́ on, Guyon, Isabelle, LeCun, Yann, Moore, C liff, Sa ̈ ckinger, Ed - uard, and Shah, Roopak. Signature verification using a siamese time delay neural network. International Jour - nal of Pattern Recognition and Artificial Intelligence, 7 (04):669 – 688, 1993. 2. Supervised Contrastive Learning by P. Kho sla, P. Teterwak, C. Wang et al, 2020 3. R.C.Gonzales, R.E.Woods, “Digital Image Processing”, 2 - nd Edition, Prentice Hall, 2002. 4. Weiyang Liu, Yandong Wen, Zhiding Yu, and Meng Yang. Large - margin softmax loss for convolutional neural networks, 2016. 5. G.V.Tcheslavski, “Morphological Image Processing: Grayscale morphology”, ELEN 4304/5365 DIP, Spring 2010. 6 Lee, Chen - Yu, Gallagher, Patrick W, and Tu, Zhuowen. Gen - eralizing pooling functions in convolutional neural networks: Mixed, gated, and tree. In AISTATS, 2016. 7 . Krizhevsky, Alex. Learning multiple layers of features from tiny images. Technical Report, 2009. 8 Szegedy, Christian, Liu, Wei, Jia, Yangqing, Sermanet, Pierre, Reed, Scott, Anguelov, Dragomir, Erhan, Dumitru, Van - houcke , Vincent, and Rabinovich, Andrew. Going deeper with convolutions. In CVPR, 2015. Fig - 2: Comparing the similarity of two signatures 9 Zeiler, Matthew D and Fergus, Rob. Stochastic pooling for regularization of deep convolutional neural networks. arXiv preprint arXiv:1301.3557, 2013. 1 0 .Zeiler,M.D., Fergu s,R.:Visualizing and understanding convolutional networks. In: Computer Vision – ECCV 2014, pp. 818 – 833. Springer (2014) 1 1 Krizhevsky, A. , Sutskever, I. , Hinton, G.E. : Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems. pp. 1097 – 1105 (2012) 1 2 Ji, S., Xu, W., Yang, M., Yu, K.: 3d convolutional neural networks for human action recognition. Pattern Analysis and Machine Intelligence, IEEE Transactions on 35(1), 221 – 231 (2013) 1 3 . Tijme n Tieleman and Geoffrey Hinton. Lecture 6.5 - rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural networks for machine learning, 4(2):26 – 31, 2012. 14. Cires ̧ an, D.C., Meier, U., Gambardella, L.M., Schmidhuber, J.: Con volutional neural network committees for handwritten character classification. In: Document Analysis and Recognition (ICDAR), 2011 International Conference on. pp. 1135 – 1139. IEEE (2011) 1 5 . Thakur, A., Prakash, A., Mishra, A. K., Goldar, A., & Sonkar, A. (2020). Facial recognition with OpenCV. In Smys, S., Tavares, J., Balas, V., & Iliyasu A. (Eds.), Computational Vision and Bio - Inspired Computing. ICCVBIC 2019. Advances in Intelligent Systems and Computing (vol 1108). Cham: Springer. https://doi.org/10.1007/978 - 3 - 030 - 37218 - 7_24 1 6 . Pramkeaw, P. (2016). The study analysis knee angle of color set detection using image processing technique. In 2016 12th International Conference on Signal - Image Tec hnology & Internet - Based Systems (SITIS) (pp. 657 – 660). Naples, 2016. https://doi.org/10.1109/sitis.2016.109 17. Dhanawade, A., Drode, A., Johnson, G., Rao A., & Upadhya, S. (2020). OpenCV based info rmation extraction from cheques. In 2020 Fourth International Conference on Computing Methodologies and Communication (ICCMC) (pp. 93 – 97). Erode, India, 2020, https://doi.org/10.1109/iccmc48092.2020.iccmc - 00018 18. V. Kumar, P. Kaware, P. Singh, R. Sonkusare and S. Kumar, "Extraction of information from bill receipts using optical character recog nition," 2020 International Conference on Smart Electronics and Communication (ICOSEC), 2020, pp. 72 - 77, doi: 10.1109/ICOSEC49089.2020.9215246. 19. R. R. Palekar, S. U. Parab, D. P. Parikh and V. N. Kamble, "Real time license plate detection using openCV a nd tesseract," 2017 International Conference on Communication and Signal Processing (ICCSP), 2017, pp. 2111 - 2115, doi: 10.1109/ICCSP.2017.8286778. 20. H. Sidhwa, S. Kulshrestha, S. Malhotra and S. Virmani, "Text Extraction from Bills and Invoices," 2018 In ternational Conference on Advances in Computing, Communication Control and Networking (ICACCCN), 2018, pp. 564 - 568, doi: 10.1109/ICACCCN.2018.8748309.