Web Application Firewall (WAF) Done by: Ahmad Jehad Haroun 201810732 Mahmoud Wajdi AlAhmad 201811207 B elal I sam A buyounes 201910186 A bedalfattaah B assam A burumman 201920297 Submitted to: Dr. Mahran M. Alzyoud Department o f Networks & Information Security Faculty o f Information Technology Al - Ahliyya Amman University Submitted i n Partial Fulfillment o f t he Requirements f or t he Degree o f Bachelor o f Networks & Information Security i n Information Technology Spring - Second Semester – 2022 2 Acknowledgment It was a great opportunity to gain a lot of experience with real time projects, followed by learning how to design and analyze real projects. Therefore, we would like to thank all the people who made this possible for students like us. Special thanks to the Graduation Project Unit for their efforts in providing us with all useful information and showing the path to student s to carry out all education periods in project design and analysis in real time. We would like to express our deepest gratitude to our Graduation Project Supervisor, Dr. Mahra Al Zayoud, for his patience and guidance throughout the semester. Furthermore , we must thank all members of the examination committee for their generous discussions and encouragement. Prof. Musleh Abu Al - Hajj, Prof. Nidal Tarab, Dr. Oraib Abu Al - Ghannam. Lecturer Sumaya Al - Khatib, Dr. Muwaffaq Abu Al - Heija, Dr. Youssef Rabaaneh, Dr. Abdul Rahman Hamza. 3 List of Tables: T ABLE 1 T OOLS , A PPS AND T ECHNOLOGY ................................ ................................ ................................ ................................ ..... 9 T ABLE 2 PRESENTS A COMPARISON B ETWEEN WAF AND IPS/IDS ................................ ................................ ................................ ... 14 T ABLE 3 D EFESE MECHANISM ................................ ................................ ................................ ................................ ................... 16 T ABLE 4 M ANAGMENT I NTERFACE ................................ ................................ ................................ ................................ ............ 16 T ABLE 5 D IFFERENCE BETWEEN SQL AND M ONGO DB ( KEYWORDS ) ................................ ................................ ................................ 23 List of figures : F IGURE 1 PROJECT TIME LINE ................................ ................................ ................................ ................................ .................... 10 F IGURE 2 H OW WAF FILTER HTTP REQUEST ................................ ................................ ................................ .............................. 13 F IGURE 3 D EFENSE M ECHANISM OF VARIOUS WAF ................................ ................................ ................................ ..................... 15 F IGURE 4 H OW THE T OOL INTERACT WITH THE T OPOLOGY ................................ ................................ ................................ ............. 21 F IGURE 5 HTTP H EADER ................................ ................................ ................................ ................................ ........................ 22 F IGURE 6 JSON BASIC FORMAT ................................ ................................ ................................ ................................ ................ 24 F IGURE 7 H OW REST API W ORK ................................ ................................ ................................ ................................ ............. 25 F IGURE 8 WAF TOOL ARCHITECTURE ................................ ................................ ................................ ................................ ......... 25 F IGURE 9 R ABBIT MQ MESSAGE BROKER OPERATION ................................ ................................ ................................ .................... 26 F IGURE 10 M IND M AP OF S ECURITY A SPECTS ................................ ................................ ................................ ............................. 28 F IGURE 11 T HE N ETWORK T OPOLOGY ................................ ................................ ................................ ................................ ....... 36 F IGURE 12 B ASIC WAF USAGE ................................ ................................ ................................ ................................ ................. 53 F IGURE 13 M ETHODS OF HOW WAF CAN WORK ................................ ................................ ................................ ......................... 54 F IGURE 14 M ETHODS OF HOW A WAF CAN BE IMPLEMENTED ................................ ................................ ................................ ....... 54 F IG URE 15 OVERVIEW OF THE TOOL DESIGN ................................ ................................ ................................ ................................ 55 F IGURE 16 DATAFLOW OF THE INTERACTION BETWEEN USER AND DEVICES SECTION IN THE WAF TOOL ................................ ................... 55 F IGURE 17 WAF TOOL MAIN DASHBOARD ................................ ................................ ................................ ................................ .. 57 F IGURE 18 UNIT TESTING ON THE DEVICE COMPONENT IN WAF TOOL AND RESULT = OK MEANS SUCCESS ................................ ............... 64 F IGURE 19 REST - API TESTING USING POSTMAN TOOL AS WE SEND GET REQUEST TO THE WAF SERVER TO GET ALL THE CAPTURES STORED IN THE DATABASE , RESULT =200 - OK. ................................ ................................ ................................ ................................ .... 65 4 Ab stract In the light of the increasing number of Internet users, there has been an increase in cyber - attacks on websites Also, w i th the addition of th e hacking and pen etration testing material that are easy - to - access T hose people called "Script Kiddies" e merged who are D o S - ing and hacking websites and IT infrastructure with already built tools. Therefore, there must be a new technology to solve the problem of secu rity for websites and servers in any industry. A product called W e b Application Firewall (WAF) appeared. WAF protects web applications through monitoring and blocking malic ious traffic through the “HTTP” and “HTTPS” protocols. WAF is implemented a s a software or appliance. In this project, w e will humbly try to tackle the problem of application layer security by building a WAF solution as a traffic control and topology monitoring software; by building the basic features of WAF using Python programming language, its libraries and other third - p arty technologies. Chapter 1: Introduction − 1.1 Problem Statement. − 1.2 Objectives. − 1.3 Overview − 1.4 Scope − 1.5 Tools, Apps and technology − Project Time line. Chapter 2: literature review − 2.1 Q/A overview about WAF − 2.2 Differences between IPS/IDS and WAF − 2.3 In - market WAF solutions − 2.4 OWASP10 − 2.5 Network automation − 2.6 HTTP (Hypertext Transfer Protocol) and its methods − 2.7 ASA (Adaptive Security Appliance) Firewall − 2.8 NoSQL databases, JSONs and REST APIs 5 − 2.9 Overview of the WAF tool − 2.10 The security aspects in the project Chapter 3: Methodology and Proposed Work − 3.1 Planning − 3.2 Design − 3.3 Implementation Chapter 4: Implementation of the Network − 4.1 Preface − 4.2 The Emulator Environment EVE - NG − 4.3 design overview − 4.4 Network topology Configurations Chapter 5: Implementation of the Code − 5.1 Introduction − 5.2 Analysis of the Tool − 5.3 System Design. − 5.4 System Implementation. − 5.5 Testing methodology − 5.6 Usage manual Chapter 6: Conclusions and Future Work − 6 .1 Conclusions − 6 .2 Future Work − 6 .3 Contribution − References 6 Chapter 1 Introduction 7 Chapter 1: Introduction − 1.1 Problem Statement. − 1.2 Objectives. − 1.3 Overview − 1.4 Scope − 1.5 Tools, Apps and technology − Project Time line. 8 1.1 Problem Statement There is no such thing as perfect security in the real world as attempts to hack into company's infrastructure and their services do exist. Although several solutions have been created to end them up , problems such as outdated code , unpatched systems, or difficulty to maintain and fix these IT - related infrastructure issues frequently occur With th e emergence of countless events of penetrating web servers and the application layer, solutions such as "WAF" appeared to reduce and even eliminate some of these problems including "OWASP 10", which is one of the most famous vulnerabilities . Many hackers a round the world use these vulnerabilities . To reach their goals and gain access to any infrastructure they want, t here is an emerging need for solutions like (WAF) which is important and effective in solving these issues C ompanies should apply it rapidly and at any cost to avoid dealing with these issues. 1.2 Objectives The primary goal of this project is to apply the skills, knowledge, and experiences gained during the bachelor's study period in the field of networks and information security, and to integ rate this knowledge with the field of web application programming as well as the research field, and to address modern topics that is relevant for the current era. Furthermore, the project is regarded as one of the research tools that aid in increasing the intellectual output of the student. As well as creating a mini - solution for what the WAF can do such as monitoring, blockin g , and discovering strange behavior. Specific goals: 1. Our solution should be working and be bug free. 2. Our s olution should have good performance 3. Our s olution s hould be easy to use. 4. Our s olution s hould be updating on any new risks and scalable 5. Implement the solution using Python language. 6. Implement several concepts such as: − Network programmin g − REST APIs − Flask − Network Automation libraries. 9 1.3 Overview WAF protects applications from malicious and unwanted internet traffic. WAF can protect any internet - facing endpoint, providing consistent rule enforcement across a Web application WAF provides you with the ability to create and manage rules for internet threats including Cross - Site Scripting (XSS), SQL Injection, and other OWASP - defined vulnerabilities. 1.4 Scope A computer network that will be created with its basic configuration, a web page and the solution that checks application layer traffic such as HTTP from the network via server and the solution bu i lt - in Python programming language 1.5 Tools, Apps and technology Table 1 Tools, Apps and Technology VMware Virtualization and cloud computing software EVE - NG Network Emulation System Linux CentOS Linux Server used in both DC& DMZ Python Programming Language Used mainly for our solution notion Used for project management Postman API testing tool VS Code Multi - purp ose IDE, easy to use for Coding Insta - gantt chart An Online tool to make Gantt Charts XMind A useful tool to make Mind Maps RabbitMQ Worker agent for process queuing protocol 10 • Project time line Figure 1 project time line 11 Chapter 2 L iterature R eview 12 Chapter 2: literature review − 2.1 Q/A overview about WAF − 2.2 Differences between IPS/IDS and WAF − 2.3 In - market WAF solutions − 2.4 OWASP10 − 2.5 Network automation − 2.6 HTTP (Hypertext Transfer Protocol) and its methods − 2.7 ASA (Adaptive Security Appliance) Firewall − 2.8 NoSQL databases, JSONs and REST APIs − 2.9 Overview of the WAF tool − 2.10 The security aspects in the project 13 2.1 Q/A overview about WAF The Web Application Firewall works as the first line of defense against the bad traffic and the web application on the webs erver, by monitoring and filtering rules. And it can be so effective against Zero Day and DD o S a ttacks WAFs may be deployed as Hardware appliance or Software tool operat ing through rules called policies, these policies allow the WAF to secure the web application from attacks. WAF keep s scanning the web application for the POST and GET requests to identify any anomalies in HTTP traffic or malicious activities. Figure 2 How WAF filter HTTP request The three main security models of WAF : 1. Whitelisting model: a. Allowing only approved traffic by the rules. b. Suitable for internal networks used by small groups. 2. Blacklisting model: a. Blocking known vulnerabilities based on static signatures b. Suitable for web applications on the public internet 3. Hybrid model: a. Configured to apply both Whitelisting and Blacklisting models. b. Can be used on both internal and public networks. 14 2.2 Difference between IPS, IDS and WAF An Intrusion Prevention System (IPS) detects anomalies in network traffic and notifies operations staff of an impending attack (IDS functionality), as well as blocking the traffic (IPS functionality). WAF almost exclusively appears to be working with web applications. WAFs must understand protocol behavior, such as HTTP GET, or FTP in addition to JavaScript, SQL, HTML, XML, Cookies Table 2 presents a comparison between WAF and IPS/IDS PARAMETER WAF IPS/IDS Abbreviation Web Application firewall Intrusion prevention System/Intrusion Detection System Functionality WAFs are designed to protect web applications/servers from web - based attacks that IPSs cannot prevent. Analyze traffic for signatures or policy violations Placement Placed before Web facing applications in web facing/DMZ zone of network Generally, on the exit entry points i.e., perimeter of network Inspection of Sessions Packets Scope HTTP/HTTPS applications Network protocols and network applications Benefits ▪ Protects Application ▪ Looks for malicious logic ▪ Enforces logic and behavior ▪ Protects OS and Application ▪ Enforces protocols ▪ Looks for malicious payloads Works at Layer 7 Layer 4 - 7 Deployment Explicit reverse proxy, Transparent mode, connected via TAP or through SPAN port Transparent mode, connected via TAP or through SPAN port Detection Algorithms ▪ Signature based ▪ Anomaly detection ▪ Heuristics ▪ Signature based ▪ Protocol based ▪ Anomaly detection ▪ Heuristics SSL Offload functionality Yes No Perform Server Load balancing Yes No Performs User authentication Yes No DDOS protection At Layer 7 Yes Functioning WAF operates at the application layer where HTML, XML, Cookies, JavaScript , ActiveX, Client requests, and Server response’s function Analyze traffic for signatures or policy violations Encryption/Decryption Supported Not Supported Inspection of Sessions where HTML, XML, Cookies, JavaScript , ActiveX, Client requests, and Server responses work Systems that analyze traffic for signatures or policy violations 15 2.3 In - market WAF solutions 1 1 - Mod Security Mod Security is an open source, free web application firewall that works on Apache system. Main features are simple filtering; regular expression - based filtering, URL encoding validation, Unicode encoding validation, auditing, null byte attack prevention, upload memory limits and server identity masking. 2 - Imperva’s Secure Sphere Imperva’s Secure Sphere, providing solutions that secure enterprise data cente rs. Secure Sphere protects proprietary information, custom business applications, and critical servers. It addresses phishing, identity theft, data theft, malicious robots, worms, denial of service, and SQL injection. It reduces web attacks, database breach, and worm infection. According to survey of Information security, Secure Sphere has high availability, preloading polices & signature and regularity compliance features. 3 - F5 – Big IP BIG - IP ASM (Application Security Manager) includes comprehensive, built - in authenticated application security policies for frequent applications as well as a regular policy - building engine that can become accustomed to application updates. This Firewall works as an appliance and provides main facilities li ke traffic monitoring and blocking. This firewall is among the top ten in the web application firewalls solutions. 4 - Barracuda Network Application Gateway Barracuda network application gateway is a commercial firewall that presents application ware traffic a dministration. Typical Barracuda Firewall functions include: a state full packet inspection firewall, IPsec VPN and intelligent traffic flow control. According to research information , Barracuda has higher capability of high availability, SLL acceleration & offloading, connections pooling, coach & compression, preloading polices & signature and regularity compliance features. 16 Table 3 Defense m echanism Table 4 Ma nagement Interface 17 2.4 The OWASP Top10 Vulnerabilities 2 The OWASP Top 10 is a list of the 10 most common web application security risks. 1: Injection Injection comes in several forms. Fundamentally injection involves inserting information that can be used to break out of the intended context of the input. Common categories of injection include Structured Query Language (SQL), NoSQL (Not only Structured Query Language), Li ghtweight Directory Access Protocol (LDAP), and operating system (OS) command injection. Injection is interesting and dangerous because it allows an attacker to potentially bypass all existing network, authentication, and authorization controls in place that protect your application. Injection can sometimes lead to data compromise or even system take‐ over. 2: Broken Authentication Broken authentication involves attack vectors such as stolen credentials, brute - force attacks, dictionary attacks, and session management attacks. if one website is hacked and a user’s password is compromised there, an attacker can use that information as part of a credential stuffing attack on different sites via password reuse. Brute - force and dictionary attacks involve repeated attempts at authentication (usually automated via botnets) using passwords from a dictionary list or via brute force. Common compensating controls include the use of Captchas, account lockout after multiple failed attempts, and enforcement of password complexity rules. Compromising session information is a different vector altogether. This might involve the execution of a Man - in - the - Middle attack to capture session data and replay that information as part of a replay attack. In some cases, session IDs are easily predictable. Compensating controls involve the use of less - predictable session identifiers and the use of digital certificates. Digital certificates help to mitigate Man - in - the - Middle attacks through encryption and through browser no tifications indicating a spoofed or untrusted certificate. This is one of the reasons why many websites have mandated an all HTTP s strategy when serving content. The technical and business impact of broken authentication can include data compromise, data l eakage, or complete system compromise if the account is a privileged system account 18 3: Sensitive Data Exposure Sensitive data exposure vulnerabilities are a result of poor data protection practices. Sensitive data can be exposed at rest and in transit. Data that is not encrypted at rest (on a drive or tape) is a prime example. Sometimes, this might involve data backups that are not encrypted and the backups fall into the wrong hands. Standard defense - in - depth strategies typically involve the use of encryption depending upon the sensitivity of the data at hand. For instance, if it is Personally Identifiable Information (PII) or PCI (credit card data), you should deploy additional protections such as encryption. Encryption is effective only if you properly manage the keys used to encrypt the data. This means that you need to implement effective key management processes and technologies. From a separation of duties perspective, the team that is responsible for managing the keys should not be directly involved with the operational management of the system itself. By separating these functions, it forces some level of collusion to take place in order to compromise the key procedurally. More modern solutions for key management involve the storage of keys in separate protected key vaults, which allow for only indirect access and usage of the key. These key vaults should be managed by a group of security administrators who are not directly involved with the day - to - day administration of the system (a database is a great example). 4: XML External Entities (XXE) XXE attacks exploit vulnerabilities in XML processing engines. You can consider it to be a form of injection attack. If an unexpected XML entity such as <! UNEXPECTED XXE SYSTEM file:///etc/passwd">]> is passed to the system and proper validation is not in place, that data can be processed in such a way that allows an attacker to break out of the context of the XML processor. Legacy Simple Object Access Protocol (SOAP) web service s prior to v1.2 are often susceptible to XXE attacks. OWASP recommends the use of less complex data formats such as JSON. If you absolutely need to use XML processing, you should update the XML process ors and libraries to the latest versions. Whitelistin g of valid inputs can help to ensure that unexpected inputs are not processed. 19 5: Broken Access Control Access controls imply authorization. In some cases, attackers can bypass application authorization mechanisms via URL manipulation, page manipulation, or custom API attacks. Attackers should not be able to access resources by simply guessing URL strings and patterns. Security through obscurity is not a viable protection pattern. All permutations of URL strings should be protected. Best prac tices can include the use of deny - all patterns and token invalidation after logout. A deny - all pattern for a firewall for instance might start with a rule statement that denies all Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) traffi c with subsequent rules that open specific ports such as port 80 for HTTP. A simple example of this type of attack might involve the manipulation of a URL such as https://mywebsite.com/products Changing this URL to https://mywebsite.com/products?purcha‐ sedby=user@email.com might allow a user to directly access resources not explicitly authorized. For an attacker, this might require only simple trial and error. An attacker can use fuzzing techniques to dis‐ cover uniden tified patterns 6: Security Misconfiguration If your house is equipped with the latest alarm systems and locks but none of them are enabled, you could say they haven’t been con‐figured properly. The same holds true with software security controls. Common attack vectors include the exploitation of known administrative accounts and default passwords, unnecessary services, and unpatched systems. 7: Cross - Site Scripting (XSS) With cross - site scripting, attackers take advantage of APIs and DOM manipulation to retrieve data from or send commands to your application. Cross - site scripting widens the attack surface for threat actors, enabling them to hijack user accounts, access browser histories, spread Trojans and worms, control browsers remotely, and more. Training developers in best practices such as data encoding and input validation reduces the likelihood of this risk. Sanitize your data by validating that it’s the content you expect for that particular field, and by encoding it for th e “endpoint” as an extra layer of protection. 20 8: Insecure Deserialization In practice, serialization involves taking data structures and sequencing them into consecutive bytes of data that can be stored in memory or on disk. Deserialization takes the seria lized data and reassembles it back into its original data structure. Examples of data structures that are commonly serialized and deserialized include JSON, XML, HTTP cookies, HTML form parameters, API authentication tokens, and Remote Procedure Calls (RPC) communications. If an attacker can influence the way that data is deserialized, they can potentially manipulate the reconstituted data structure in a manner that compromises the integrity of the application. To pre‐ vent deserialization attacks, it i s best to not accept serialized objects from untrusted sources. 9: Using Components with Known Vulnerabilities When application developers use code libraries with known vulnerabilities, they are making the application directly vulnerable. Given the exponential increase in the amount of third - party code libraries that developers use, this becomes a nagging issue in development and production environments. A best practice is to continually “repave” or redeploy an application’s micro services on a frequ ent basis, which incorporate the latest, patched versions of affected libraries. This works great in mature DevOps environments, but many shops have not reached this level of maturity. At a minimum the security operations team should be scanning production applications for vulnerabilities and patching the application on a proactive basis. 10: Insufficient Logging and Monitoring (new) Even though logging is a detective control in nature, its absence leads to a lack of visibility as it relates to threats. Ap plications should be sufficiently instrumented so that security - related events are captured and logged as needed. This allows security operations teams to monitor and correlate this information with other security and net‐ work events to facilitate proper threat identification and incident response procedures 2.5 Network automation Network programmability is a trend, enhanced and inspired by Software Defined Networks, that are based on scripting methods and standard programming languages used for