The World-Wide Web - What Is It, Where Does It Come From, What Went Wrong?
What Is It, Where Does It Come From, What Went Wrong?
- The Internet
- The World Wide Web
- Phases of The Web
- Clients and Servers
- Layered Architecture
- Hypertext Transfer Protocol
- Front-End Technologies
- HTML in 4 slides
- CSS in 3 slides
- Back-End Technologies
- Databases
- Monolithic vs Microservices Architectures
- Conclusions
The Internet
Definition
The Internet is the global interconnected network of computers that communicate via a set of standardized protocols.
- It is sometimes referred to as a “network of networks.”
Connective tissue
Copper wires and radio signals account for a smaller fraction of the total distance traveled by data on the Internet.
The vast expanses of the Internet’s reach are today enabled primarily by fiber optic cables that run locally, regionally, and across ocean floors to connect continents - see a map of submarine cables.
Origins
The Internet evolved over decades, funded primarily by the US Department of Defense
-
1960s - US Department of Defense networking projects explored fault-tolerant networks resistent to nuclear attack
-
1970s-80s - The ARPANET, a mostly regional network, attempted to improve interoperability among different computer systems
-
1980s-90s - This evolved into the Internet, a global network of interoperable computers
Ownership
The “backbone” of networking equipment that fuels the Internet belongs to a patchwork of different interests:
-
telecommunications companies and other private corporations
-
federal, state, and local government
-
academia
Cooperation
Given the patchwork of operators controlling different parts, cooperation is key.
-
Organizations of approximately equal size that provide key parts of the Internet backbone usually abide by a “free peering” agreement, whereby they agree to relay each others’ data across their own network hardware for free.
-
Organizations of different sizes who need to cooperate often work out payment agreements to let traffic from one travel over the other’s equipment
Control
The Internet, for better or worse, has been much touted as a disruptive force in society.
-
A decade or two ago, many saw the Internet as a democratizing force for oppressed peoples everywhere.
-
Today, some point out a more complicated situation where information and access to that information is controlled by corporate and government interests.
-
The Internet, where anyone anywhere can share information, steal information, surveil and influence populations on the other side of the globe, is increasingly the site of a struggle for information dominance.
The World Wide Web
Definition
The World Wide Web is one use of the Internet to publish and browse hypertext documents.
Hypertext
“Hypertext” means documents which reference and link to one-another.
-
The hypertext concept was first popularized by Dr. Vannevar Bush in a 1945 article in The Atlantic titled, “As We May Think”.
-
Vannevar Bush was then the Director of the Office of Scientific Research and Development for the US military.
-
After World War II, where he managed miliatry scientific research including the development of the atomic bomb, Bush turned his sights on the problem of information overload.
Origin
The World Wide Web was invented in 1989-1990 by a British researcher at CERN named Tim Berners-Lee.
-
Berners-Lee acknowledged that his invention was really a mashup of existing concepts, including hypertext, the Internet, and multi-font text.
-
Inventing The Web required him to think at a higher level of abstraction and generalization to envision the value of this combination.
-
He then invented Hypertext Markup Language (HTML) that is used to write web documents, and created the first web browser, the first web server, and invented the Hypertext Transfer Protocol (HTTP) that allows web browsers and web servers to communicate with a specific set of commands.
-
He released his code free and open source, of course.
-
Listen to him discuss the current state of his invention.
Definition (revisited)
In the past, it was simple to define the World Wide Web.
-
the use of a web browser to browse hypertext documents on the Internet
-
use of the HTTP protocol by clients and servers to request and receive information
Definition (revisited)
Today, the situation is a bit more blurry.
-
Many apps besides web browsers now allow users to publish and read public and private hypertext documents over the Internet
-
HTTP is now used to do much more than transferring hypertext documents - it is being used for all kinds of file transfers, messaging, virtual meetings, and more.
-
other protocols, such as WebSockets, can now be used to transfer hypertext documents
Phases of The Web
Web 1.0
In the early days of The Web, pages contained primarily static content with limited interactivity.
- Users were mostly passive consumers of content.
Web 2.0
Later, at around the start of the millenium, many web pages began to solicit user-generated content, provide social networking features, and the massive data collection of user behavior began.
- Users began to actively participate in content creation, knowingly or not.
Web3
As the ill effects of the centralized architecture of The Web became clear, interest in alternative, decentralized, privacy-supporting architectures increased. Especially after the inventions of Bitcoin, Ethereum, the InterPlanetary File System (IPFS), Mastodon and others in the 2010’s, the vision of a decentralized open Web seemed to be within reach.
- Users began to communicate directly with one-another as peers using distributed, decentralized protocols, controlling their own data, rather than through central servers that filtered their access to information and surveilling their every microservice.
Clients and servers
Definitions
-
a client is any machine on a network that is requesting data from a server.
-
a server is any machine that is responding to a request for data from a client machine.
-
a single machine may sometimes act the role of a client in one interaction and the role of a server in another.
-
the terms client or server can also be used to refer to the specific software on the client or server machine that is handling the communication in question.
Requests
A client makes a request to a server.
Responses
The server responds to the client.
Centralized topology
A single server typically handles requests from many clients.
Peer-to-peer
In peer-to-peer networks, such as BitTorrent and Bitcoin, any machines on the network may simultaneously act as a client in one exchange while playing the role of a server in another.
Layered Architecture
Overview
-
communication between clients and servers (or among peers) over the Internet is standardized
-
a set of Internet Protocols control the languages used in these interactions
-
these protocols are designed in a layered architecture, where protocols in each layer depend and upon the protocols in the layer directly below it.
Diagram of layers
-
HTTP is concerned with properly generating requests and responses and sending them via TCP
-
TCP is used to ensure successful transmmission of request/response data from one computer to another using IP
-
IP makes sure data packets are packaged and addressed correctly
-
Various networking protocols are then used to pass the data along local and regional networks
Hypertext Transfer Protocol
Standardized requests and responses
The HTTP specification indicates
- how Web clients can make requests to Web servers
- how Web servers can respond to those requests from Web clients
Plain text
HTTP request and response commands are sent as plain text over the network, following a specific format.
- for added security, the HTTPS “secure” extension of HTTP encrypts the request and response communication while in transit
Request types
The two most widely used request types allowed by the HTTP specification
-
GET - a web client requests that the web server respond by giving it a specific information resource, such as a web page
-
POST - a web client supplies some data to the web server along with the request with the intention that the server save that data, or at least use it to change its behavior in some way
-
other HTTP request types do exist, but are less widely used.
Response codes
Web server responses include a status code, indicating the success or failure of the client’s request.
- 200 to 299: success
- 400 to 499: client error
- 500 to 599: server error
- and more…
Example of a web client GET request
GET /some_document.html HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE5.01; Windows NT)
Host: knowledge.kitchen
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Example of a web client POST request
POST /some_document.html HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE5.01; Windows NT)
Host: knowledge.kitchen
Content-Type: application/x-www-form-urlencoded
Content-Length: 40
Accept-Language: en-us
Accept-Encoding: gzip, deflate
name=Foo+Barstein&topic=web+architecture
Example of a successsful web server response
HTTP/1.1 200 OK
Date: Mon, 27 Jul 2022 12:28:53 GMT
Server: Apache/2.2.14 (Win32)
Last-Modified: Wed, 22 Jul 2009 19:15:56 GMT
Content-Length: 88
Content-Type: text/html
Connection: Closed
<html>
<body>
<h1>Hello, World!</h1>
<p>How are you?</p>
</body>
</html>
Front-End Technologies
Overview
Front-end technologies are those languages or programs which are interpreted and executed by the client, typically a web browser.
Separation of concerns
Any web browser understands the three core front-end web technologies:
- Hypertext Markup Language (HTML) - controls the content present on the web page
- Cascading Style Sheets (CSS) - controls the visual style of the content on the web page
- Javascript (JS) - controls the iteractive behavior of the web page (how does the page change when a user clicks, types, moves them mouse, etc)
HTML
Used to indicate the content and semantic meaning of the content on a web page.
<article>
<h1>Jaberwocky</h1>
<p id="p1">
Twas brillig and the slithy toves did gyre and gimble in the wabe:
</p>
<a id="more" href="#">click to see more</a>
<p id="p2">All mimsy were the borogoves, and the mome raths outgrabe.</p>
</article>
CSS
Used to indicate the style and presentation of a web page.
/* styles for the second paragraph */
#p2 {
display: none; /* hide the second stanza by default */
}
/* styles for the link */
#more {
color: red; /* make the text red */
text-decoration: none; /* remove the default underline from the link */
}
Javascript
Used to indicate interactive behaviors of a web page.
// when the link is clicked...
$("#more").click(function () {
$("#p2").show() // make the second stanza appear
})
It’s up to a client’s interpretation
A web browser client receives the HTML, CSS, and Javascript code from the server, and then interprets it.
Frameworks
Front-end frameworks attempt to simplify and streamline the process of creating front-end interfaces.
-
Bootstrap - a framework ,originally by Twitter, with readymade CSS and Javascript code that handles many common interface design needs
-
Material UI - a competitor to Bootstrap, by Google, specifically design to modularly fit into any project using React
-
React - a framework, by Facebook, that attempts to abstract and streamline the process of generating the HTML, CSS, and Javascript code for a web or mobile app; allows for easy sharing among projects of modular “components” and behaviors
-
AngularJS - a competitor to React, by Google
HTML In 4 Slides
Semantic markup
For any given web page, HTML code indicates the type of content to show as well as each piece of content’s purpose, or semantic meaning.
- A level 1 heading:
<h1>Puppies For Sale</h1>
- A paragraph:
<p>Buy now while supplies last!</p>
- An image of a lonely puppy:
<img src="images/lonely_puppy.jpg" />
- An article with a level 1 heading, an image of a sad puppy, and a paragraph with a link to another web page:
<article>
<h1>Puppies For Sale</h1>
<img src="images/lonely_puppy.jpg" />
<p>
Buy now while supplies last!
<a href="puppies.html">Learn more</a>
</p>
</article>
Markup structure
Many web sites see the same basic content structures repeated across multiple pages, or repeated within a single page. Try it!
<section>
<!-- the first article -->
<article>
<h1>Puppies For Sale</h1>
<img src="images/lonely_puppy.jpg" />
<p>
Buy now while supplies last!
<a href="puppies.html">Learn more</a>
</p>
</article>
<!-- the second article... notice its structural similarity to the first -->
<article>
<h1>Kittens For Sale</h1>
<img src="images/sad_kitten.jpg" />
<p>
Get 'em while they're hot!
<a href="kittens.html">Learn more</a>
</p>
</article>
</section>
Forms
HTML allows web visitors to enter data into forms - there are a variety of different user interface components specific to forms. For example:
<form action="/send" method="POST">
<label for="fname">Your name:</label>
<input type="text" name="fname" />
<!-- a one-line text field for their email address -->
<label for="femail">Your email:</label>
<input type="email" name="femail" />
<!-- a multi-line input for their comment -->
<label for="fmessage">Your comment:</label>
<textarea name="fmessage" ></textarea>
<!-- submit button with the words, 'Send to us!' on it -->
<input type="submit" value="Send to us!" />
</form>
Forms (continued)
Some important and required form tag attributes:
<form action="/send" method="POST">...</form>
-
action="/send"
indicates that when the user submits the form, the web browser will send a request to the server’s/send
route. Presumably the server is set up properly to receive the form data at this route and do something useful with it. -
method="POST"
indicates that the web browser will submit the data as an HTTPPOST
to the server, rather than an HTTPGET
request, which is the browser’s default, remember?
That’s it.
What you just learned is how to code for content that will be displayed in the body
of a web page - the part of an HTML document that represents the visible content.
CSS
Adding style
For any given web page, CSS code indicates the style to use for the content indicated by the HTML.
- Give all level 2 headings green text:
h2 { color: green; }
- Give all paragraphs a pink background:
p { background-color: pink; }
- Set the dimensions of all images:
img { width: 200px; height: 300px; }
- Give all articles a thick pink dotted border, a bold
serif
font, and lots of space around the contents inside them:
article {
border: 10px dotted pink;
font-family: georgia;
font-weight: bold;
padding: 20px;
}
Selecting one or more elements
Let’s say you had four paragraphs, and wanted to style only two of them the same. Mark them with a class
attribute in the HTML. If you want to uniquely style only one, mark it with an id
attribute:
<p>Lorem ipsum dolor sit amet.</p>
<p class="my_favorite_paragraphs">Ipsum dolor sit amet lorem.</p>
<p id="a_really_special_paragraph">Dolor sit amet lorem ipsum.</p>
<p class="my_favorite_paragraphs">Ipsum dolor sit amet lorem.</p>
And select only the paragraphs that match these attributes in the CSS:
p.my_favorite_paragraphs {
box-shadow: 5px 5px 10px; /* a drop shadow */
}
p#a_really_special_paragraph {
font-size: 24px; /* relatively big */
}
Positioning elements and creating layouts
That’s almost it. But there is one very important part of CSS code that we will not address here is how you place elements onto the web page in the location where you want them to go.
There are a variety of ways… the latest greatest is to CSS Flexbox - see a simple example - flexbox is how layouts should be done with React.
- First learn more about basic CSS.
- Then learn more about layouts with CSS flexbox.
Back-End Technologies
Overview
Back-end technologies are those languages or programs used to control the behavior of the server and how it responds to various kinds of client requests.
Servers control what the client receives
The server can be programmed to modify the HTML, CSS, and Javascript code that is sent to the client in response to the request.
Examples
For example, back-end programming on the server-side might allow a web server to…
-
Send a Spanish version of a web page to a user whose browser’s HTTP request said it prefers Spanish, and an English version of the same page to a browser with English as its language setting.
-
Send a mobile version of a web page to a user whose browser’s HTTP request said it is running on a mobile device, and a desktop version of the same page to a browser whose HTTP request says it’s on a desktop computer.
-
Send the client a web page that automatically includes today’s news today and tomorrow’s news tomorrow, rather than sending the client the same news articles every day.
-
Allow a server to “remember” what products each user adds into their shopping cart; and serve all users the same shopping cart page, but with their own cart’s products in it, not someone else’s.
Popular back-end programming languages
Just about any high-level programaming language of note can be used to control how a web server responds to requests.
Frameworks
The choice of frameworks to simplify the process of writing server-side code depends on which server-side programming language is being used.
- Node.js (server-side Javascript) usually use the Express framework
- Ruby projects usually use the Rails framework
- Python projects usually use the Django or Flask frameworks
- PHP projects often use Laravel or Sympfony frameworks
- C# projects usually use the .NET framework
- ASP projects usually use the .NET framework
Example
This example, written in Python with flask
and pymongo
modules, would save data submitted by an HTML form into a MongoDB database document.
@app.route('/save', methods=['POST'])
def save_message():
name = request.form['fname'] # retrieve the data the user entered into the fname field
name = request.form['femail'] # retrieve the data the user entered into the femail field
message = request.form['fmessage'] # retrieve the data the user entered into the fmessage field
# create a new document with the data the user entered, plus the current date & time
doc_to_insert = {
"name": name,
"email": email,
"message": message,
"created_at": datetime.datetime.utcnow() # the time right now, in UTC time
}
collection = app.db["exampleapp"] # select the collection where we want to save this document
mongoid = collection.insert_one(doc_to_insert) # insert a new document, store the new document's _id value
return render_template('show.html', mongoid=mongoid, doc=doc_to_insert) # render an HTML page with the new post displayed
Databases
Overview
As you undoubtedly know, a database is any technology used to store data
- fast storage and retrieval
- easy integration with back-end technologies and programming languages
Data flow
The server-side code usually communicates with the database to save new data sent from the client, or to retrieve existing data and use it to determine how to respond.
Popular back-end databases for web sites and apps
Relational databases based on the SQL language have traditionally dominated since the 1970’s.
- Oracle
- MS SQL
- PostgreSQL
- MySQL
- Amazon Aurora
- … and more
Popular back-end databases for web sites and apps
However NoSQL databases are now surging, due to the changing expectations of web and mobile app developers and those using cloud-based infrastructure.
- MongoDB
- Redis
- Google Firebase
- Amazon DynamoDB
- … and more
Serverless databases
A growing market is the “serverless database”, where the server provides the initial front-end code to the client, but further data is sent/received by front-end Javascript directly to/from the database.
Front-end data storage for web apps
There are some limited data storage capabilities of web browsers, where data used by a web site is stored locally on the client machine.
-
Cookies - often used to store tracking codes that are secretly passed to the server in an HTTP header with every request
-
LocalStorage - often used to store larger block of data, such as web pagee content that can be displayed even when browser is offline
Monolithic vs Microservices Architectures
Monolithic architecture
What we have been discussing until now is termed a monolithic architecture
-
One server directly handles all back-end tasks
-
The “traditional” architecture for web and mobile apps
-
Over time, the codebase tends to grow larger, more complex and tangled and becomes difficult to maintain
Microservices architecture
A growing trend for scalable apps is the microservices architecture.
Microservices architecture
-
Multiple single-purpose servers each handle a single discrete duty of the back-end.
-
Each microservice has its own codebase maintained independently of the others.
-
There is no need for all microservices to use the same technology stack… not stuck with legacy codebase.
-
A “gateway API” server coordinates the calls to the microservices and provides a unified API to the client.
Conclusions
-
You now have some understanding of the internet, the Web, HTTP requests and responses and how various front- and back-end technologies are used.
-
Thank you. Bye