In a previous article, I expanded upon how the internet works using a popular conceptual model called the OSI model. While the internet and world wide web are used interchangeably, they refer to different things. The world wide web (www) is a subset of Internet, and is a collection of linked documents and resources accessed via your browser.

The purpose for this exercise is for web developers to have a guiding list that ensures the applications they develop are robust, while also ensuring new learners have a comprehensive ‘big picture’ of the playing field.

I make a clear distinction here — in the internet article, I expand mainly on the architecture and physical aspects of the internet. Here, we are talking about the software aspect of internet, under the guise of world wide web, and we are specifically interested in how it gives rise to your modern web applications.

Acknowledging this distinction is pivotal in comprehending the intricate framework underpinning modern web development — a topic I’m eager to delve deeper into in forthcoming pieces. However, it is also worthwhile to note that the distinction is difficult at some places, so hardware concepts can also be explained.

A. FRONTEND

The frontend of a website is the part which the user interacts with, and the one that is visible to the user. It includes all the presentation of a website as well as all interactivity happening on the user’s end.

HTML and CSS

While HTML and HTTP may sound similar, they serve very different roles. If HTTP is the messenger, HTML is the message that is delivered. HyperText Markup Language (HTML) is a language which defines the structure of the webpage. Cascading Style Sheet (CSS) is a language that defines how the website looks. Both HTML and CSS work together to give you a webpage on the world wide web.

JavaScript

JavaScript allows for interactive websites, such as displaying alert messages and changing messages on the screen without refreshing the webpage.

Key features that make JavaScript essential for web development are:

1. Client-side interactivity: Allows for dynamic interactions that are instant and real-time, resulting in an engaging user interface.
2. DOM Manipulation: HTML structures the webpage in a Document Object Model (DOM), which can be visualized as a tree-like structure. With JavaScript, developers can change parts of a web page on the go using DOM manipulation. This means they can update text, change colors, or rearrange items based on what you do or click on the site.
3. Event Handling: JavaScript responds to things you do on a site, like clicks or mouse moves. It powers interactive stuff like dropdown lists, photo sliders, and checks when you fill out a form. It’s like the senses of a web page, noticing when you interact with it.
4. Asynchronous Operations: JavaScript can do multiple things at once without waiting for one to finish, thanks to features such as callbacks, promises, and async/wait. This results in a faster and smoother experience for users.
5. Cross-Browser Compatibility: JavaScript is supported by all major web browsers across several platforms.
6. Frameworks and Libraries: Having a rich ecosystem of frameworks and libraries such as React, web3, Angular, Vue.js simplify web development. These frameworks help provide reusable components for efficiently building complex web applications.
7. Server-Side Development: With Node.js, JavaScript can be used for server-side (backend) development in addition to frontend development.

B. NETWORKING

Applications on the internet require function on an orchestra of well coordinated networking technologies which allow for a reliable and fast experience.

Client-Server Model

Central to the understanding of the web is is client-server architecture. Using a restaurant analogy, users on the web are the customers (clients) of the restaurant. The client requests the kitchen (server) using HTTPS (waiter).

HTTPS

The HyperText Transfer Protocol (HTTP) is a TCP/IP based protocol on the Application layer that allows for secure (in the case of HTTPS) communication between a client and a server. HTTPS is a stateless protocol and does not keep any data (states) between requests.

Credit: https://developer.mozilla.org/en-US/docs/Web/HTTP/Overview

HTTP is based on methods, often verbs like GET and POST or nouns like OPTIONS and HEAD, and are operations that are performed by clients. These methods stipulate the specific action a client wishes to execute. Whether it’s retrieving a resource with GET or submitting an HTML form via POST, these methods facilitate varied operations tailored to diverse needs. As explained previously, HTTP, especially HTTPS, operates in a stateless manner, refraining from retaining any data between successive requests.

WebSockets

Provides full-duplex communication channels over a single TCP connection. HTTP works on a stateless, request-response model whereby a client sends a request, waits, and gets back a response from the server. Websockets establish a real-time communication channel between the two parties, resulting in an experience where there is lower latency and less overhead.

Content Delivery Network

Users expect fast load times and responsiveness from the modern web and mobile applications, and therefore, content delivery networks have evolved as a network of strategically placed servers around the world, geographically closer to users. Due to this decentralized structure, users experience better performance and organizations achieve better fault tolerance, scalability and reliability.

Examples: Amazon CloudFront, Fastly

Domain Name Service

Imagine having to type ‘142.250.65.78’ instead of ‘www.google.com’ every time you had to access Google search. Every website you visit on the internet has an IP address like the one mentioned here, which the browser has to access. However, the Domain Name Service stores all this information for you, so you just have to type ‘www.google.com’ and the DNS server responds accordingly.

Examples: Cloudflare DNS, Cisco OpenDNS

Hosting

All web content needs somewhere to ‘live’, and servers are usually computers where it lives. We as clients access that content using the internet. Anyone who needs to make a web service or website available on the internet needs to host it on a server, and hosting services provide just that. Additionally, modern hosting services provide security and scalability services as well.

Examples: DigitalOcean, HostGator

Browsers

Browsers help users view the websites in a user-friendly manner, as they are written primarily in specific coding langauges such as HTML, CSS, and Javascript. A browser interprets these languages and displays all the elements of the web page along with all the text, images, videos, and interactive elements.

Examples: Google Chrome, Firefox, Safari

Proxy Servers

Proxy servers act as intermediaries between a client and the internet, offering both improved security and increased functionality. When a client requests a webpage, the proxy server makes the request on behalf of the client, receives the response, and then forwards it back to the client. This process ensures that the client’s IP address remains hidden, enhancing privacy. Proxy servers also cache frequently accessed content, speeding up subsequent requests for the same content. Additionally, they can enforce access policies and log web requests, playing a crucial role in network security and data analysis.

Examples: HideMyAss, NordVPN

Forward Proxy

A forward proxy, positioned at the client’s end of the network, primarily serves to direct outgoing requests. When a client sends a request, the forward proxy receives it and decides whether to permit it based on preset rules. It can also modify the request as needed. This type of proxy is commonly used in organizations to monitor and control internet usage, as it can block access to certain sites and provide anonymity for internal network users.

Examples: Fiddler, Charles Proxy

Reverse Proxy

A reverse proxy, such as NGINX, operates at the server’s end, managing incoming requests to those servers. It acts as a gateway, directing client requests to the appropriate server. One key advantage of a reverse proxy like NGINX is its ability to balance load among multiple servers, enhancing the performance and reliability of web applications. NGINX can also serve cached content, handle SSL termination, and provide additional security layers against malicious attacks. Reverse proxies are also very useful in directing traffic in a microservices-based setup.

Examples: Apache Traffic Server, HAProxy

C. Backend

Just like the website’s frontend, the backend is as crucial. It contains all the logic and technicalities that the frontend displays and it helps the server process the requests you make.

Databases

Data generated on the website needs to be stored permanently such that the server can access it and display it to you when you are back again. Examples of such Data are your username, password, transaction history, etc. Additionally, it is not just enough to store the data — it should be stored in a manner that it is easily accessed, managed, and updated. Selecting the right database type and service is an architectural science of its own.

Examples: MySQL, PostgreSQL, MongoDB, and SQL Server.

Server-side Scripting

This code helps manage the logic of a website, from user authentication to data processing. It is essentially the ‘brains’ behind the scenes that responds to your requests and customizes your experience. It knows when to access what, and from where.

Examples: Node.js (using JavaScript), Python, Ruby, PHP, or Java.

Application Programming Interfaces (APIs)

Tools and protocols that allow different software applications to communicate with each other. For example, a weather app on your phone might pull data from a weather API hosted on a server, or a ride-sharing app might pull data from a maps API.

REST API

REST (Representational State Transfer) API is a set of principles that provide a way for systems to communicate over the internet. It uses standard HTTP methods like GET, POST, PUT, and DELETE. APIs built following REST principles are known as RESTful APIs. They allow for interaction with RESTful web services, enabling actions like retrieving or updating data. REST APIs are stateless, meaning each request from a client to a server must contain all the information needed to understand and complete the request. They are widely used due to their simplicity, scalability, and compatibility with web technologies.

Examples: Twitter API, Google Maps API, and GitHub API.

Server Software

Servers can run various software to manage and deliver content, and their selection depends on the requirements of the application, like performance, security, and scalability. Common examples of server software are Apache, known for its robustness, Nginx, popular for its high performance and low resource usage, and Microsoft IIS, used mainly in Windows environments.

Examples: Apache, Nginx, and Microsoft IIS.

Middleware

Middleware is the software layer that sits between the operating system or database and the applications, especially in a networked environment. It facilitates communication and data management for distributed applications. It can include web servers, application servers, content management systems, and other tools that support application development and delivery. Middleware is crucial for complex software development, as it enables different components of an application to communicate and work together efficiently.

Examples: IBM WebSphere, Oracle Middleware, Red Hat JBoss Middleware

Authentication & Authorization Systems

These are critical components in any web application, designed to manage user accounts and roles. Authentication verifies the identity of a user, while authorization determines their access rights within the system. These systems ensure that only authenticated and authorized users can access certain data and functionalities, enhancing security. They can range from simple login systems to complex frameworks handling multi-factor authentication and sophisticated role-based access control.

Examples: OAuth, JWT (JSON Web Tokens), LDAP (Lightweight Directory Access Protocol).

D. Performance Optimization Tools

Performance optimization is a critical aspect of web development, ensuring that applications run efficiently, load quickly, and provide a seamless user experience. By using various tools and techniques, developers can significantly enhance the performance of both frontend and backend components of a website or application.

Caching Systems

Caching systems are used to temporarily store frequently accessed data to reduce the time and resources needed to fetch this data from the original source. By serving data from the cache, response times can be drastically reduced, leading to faster loading times and improved overall performance. Common caching systems include Redis and Memcached, which are highly efficient for storing key-value pairs in memory. They are particularly useful in scenarios where the same data is read frequently but updated less often.

Examples: Redis, Memcached.

Web Optimizers

Web optimizers are tools that help in optimizing frontend code, making web pages load faster and run more efficiently. These tools typically perform tasks like minifying JavaScript and CSS files (reducing their size by removing unnecessary characters), optimizing images, and bundling multiple files into a single file to reduce the number of server requests. Webpack and Gulp are popular examples of such tools. They automate the process of code optimization and are integral in modern web development workflows.

Examples: Webpack, Gulp.

Load Balancers

Load balancers play a crucial role in handling incoming web traffic by distributing it across multiple servers. This not only ensures that no single server is overwhelmed by requests, but also helps in achieving high availability and redundancy. If one server fails, the load balancer redirects traffic to the remaining operational servers, thus maintaining the website’s functionality. Load balancers can be software-based or hardware-based and are essential in handling high traffic volumes efficiently.

Examples: NGINX, HAProxy, AWS Elastic Load Balancing.

Content Delivery Networks (CDNs)

Content Delivery Networks are networks of servers strategically located around the world to deliver web content more efficiently to users based on their geographic location. CDNs store cached versions of static content like images, CSS, and JavaScript files, so they are closer to the user, reducing latency and improving load times. CDNs not only enhance performance but also help to offload traffic from the origin server, contributing to better scalability and reliability of web applications.

Examples: Cloudflare, Akamai, Amazon CloudFront.

Database Optimization Tools

Optimizing the performance of databases is crucial in ensuring quick data retrieval and efficient transaction processing. Tools for database optimization can include query optimizers, index tuning tools, and performance monitoring solutions. They help in identifying bottlenecks, optimizing database queries, and ensuring that the database schema is designed for efficiency.

Examples: Oracle SQL Tuning Advisor, MySQL Query Optimizer, PgBouncer.

Profiling and Monitoring Tools

Profiling and monitoring tools are essential for identifying performance bottlenecks and optimizing resource usage in applications. These tools provide insights into various aspects like CPU usage, memory consumption, and request processing times, enabling developers to pinpoint inefficiencies and improve the overall performance of the application.

Examples: New Relic, Datadog, Grafana.

E. DevOps

DevOps, a blend of development and operations, is an approach that focuses on collaboration, automation, and integration between software developers and IT professionals. It aims to shorten the development life cycle and provide continuous delivery with high software quality. In the realm of DevOps, several tools and technologies play crucial roles in achieving these objectives.

Version Control Systems

Version control systems are fundamental to DevOps practices. They allow developers to manage and track changes to the codebase, collaborate with team members, and maintain a history of modifications. This is crucial for coordinating work among team members, reverting to previous versions in case of errors, and understanding the evolution of a project over time. Git is the most widely used version control system today, known for its flexibility and efficiency in handling both small and large projects.

Examples: Git, Subversion (SVN), Mercurial.

Web Servers

Web servers are the backbone of internet services, responsible for serving web pages to users upon request. They handle incoming HTTP requests, process them, and send the appropriate content back to the client. In a DevOps environment, the choice of web server can impact the performance, scalability, and security of web applications. Apache and Nginx are two popular web servers, each with its own strengths; Apache is known for its robustness and rich feature set, while Nginx is celebrated for its high performance and low resource consumption.

Examples: Apache, Nginx, Microsoft IIS.

Cloud Services

Cloud services have become a staple in DevOps due to their flexibility, scalability, and wide range of services. They provide on-demand computing resources and services, from hosting and computing power to databases and machine learning, often with pay-as-you-go pricing. This allows DevOps teams to quickly provision resources as needed, scale up or down based on demand, and focus on innovation rather than infrastructure management. Leading cloud service providers include AWS, Google Cloud, and Microsoft Azure, each offering a rich ecosystem of services.

Examples: Amazon Web Services (AWS), Google Cloud Platform (GCP), Microsoft Azure.

Containerization Tools

Containerization is a key technology in DevOps for packaging and deploying applications in a consistent, isolated, and efficient manner. Containers encapsulate an application and its dependencies into a single, portable unit that can run consistently across different environments. Docker is the most popular containerization platform, known for its simplicity and efficiency. Kubernetes, on the other hand, is a container orchestration system that manages and scales containerized applications on clusters of physical or virtual machines.

Examples: Docker, Kubernetes.

CI/CD Tools

Continuous Integration (CI) and Continuous Deployment (CD) are practices that automate the integration of code changes from multiple contributors into a single software project, and the delivery of this code into production environments. CI/CD tools automate these processes, enabling frequent, reliable, and efficient release of software changes. They help in detecting and addressing bugs quickly, improving software quality, and reducing the time taken to deliver updates. These tools are essential in maintaining a steady flow of software development, testing, and deployment.

Examples: Jenkins, GitLab CI/CD, Travis CI, CircleCI, Bamboo.

F. Logging and Reporting

Logging and reporting are crucial components in the development and maintenance of software applications. They provide insights into the application’s performance, help identify and diagnose issues, and offer valuable data for decision-making. Effective logging and reporting tools are essential for monitoring application health, understanding user interactions, and ensuring security and compliance.

Logging Tools

Logging tools are used to record events that occur within an application or system. These logs provide a chronological record of events, errors, and transactions, which are invaluable for troubleshooting issues and understanding system behavior. Log data can include everything from user actions to system errors, and from performance metrics to security alerts. Popular logging tools like Logstash and Splunk allow for efficient collection, processing, and analysis of log data, offering a comprehensive view of an application’s operations.

Examples: Logstash, Splunk, ELK Stack (Elasticsearch, Logstash, Kibana).

Error Tracking and Exception Monitoring

Error tracking and exception monitoring tools automatically detect and report errors that occur in a software application. These tools capture detailed information about exceptions and crashes, enabling developers to quickly identify and address issues. They often provide features like stack traces, error aggregation, and alerting, which are critical for maintaining the health and performance of applications. Sentry and New Relic are well-known for their robust error tracking and real-time monitoring capabilities.

Examples: Sentry, New Relic, Raygun.

Application Performance Monitoring (APM)

Application Performance Monitoring tools help in assessing the performance of applications by tracking various metrics like response times, transaction volumes, and error rates. They provide real-time visibility into the application’s performance, helping to identify bottlenecks and optimize efficiency. APM tools are essential for ensuring that applications meet performance standards and provide a good user experience.

Examples: Datadog, AppDynamics, Dynatrace.

Reporting and Analytics

Reporting and analytics tools are used to convert data into actionable insights. They gather data from various sources, including logs, user interactions, and performance metrics, to generate reports that aid in decision-making. These tools can provide detailed analysis on usage patterns, customer behavior, and system performance, which is crucial for strategic planning and continuous improvement.

Examples: Google Analytics, Tableau, Microsoft Power BI.

Security Information and Event Management (SIEM)

Security Information and Event Management tools are essential for managing security events and ensuring compliance with regulatory standards. They provide real-time analysis of security alerts generated by applications and network hardware. SIEM tools help in detecting, analyzing, and responding to security threats, ensuring that sensitive data is protected. Products like Splunk Enterprise Security and IBM QRadar are popular choices for their advanced security analytics and threat management features.

Examples: Splunk Enterprise Security, IBM QRadar, AlienVault.