The components of interactive application design

From Knowledge Kitchen
Jump to navigation Jump to search


A sweeping, non-linear, free-associative discussion on all things appy and interactive. Tentative discussion points follow.


Typical uni-dimensional application

Ye ol' input->processing->output paradigm.

Input

Some kind of source material of interest is supplied to the app

.

  • some input is often in the form of a dynamic parameter that you cannot entirely predict
  • other input is static, fixed, never changing

Common sources of input

  • buttons
  • mice / trackpads
  • microphone
  • camera
  • touch screen
  • windmill
  • radio transceiver
  • database
  • API
  • Arduino/Pi circuit with sensor
    • air pollution
    • flex
    • knob
    • implanted devices
    • etc

Example of a mouse as input:

Processing

The app performs some sort of magic based in-part on that input

. The variety is limitless.

Output

That magic results in something of interest or value that makes the whole endeavor worthwhile


  • motor pleasantly whirs
  • speakers produce fanastastic sound
  • digital display shows something amazing
  • an app or web site is updated to show breathtakingly original content
  • lights flash in brilliant glittering pattern

Example of glittery output

System characteristics

Open or closed

  • Is the app's universe a closed system with no external inputs, data sources, or outputs?
  • or does it break out of its own box and interact with other systems?

Closed systems are easier to design and less prone to error and need less upkeep. However, open systems are, by nature, more open-ended.

Deterministic or stochastic

  • In deterministic systems, the output of the system is fully determined by the input parameter values and the initial conditions. The app functions like a black box. These are somewhat predictable.
  • Stochastic systems possess some inherent randomness. The same set of parameter values and initial conditions will lead to an ensemble of different outputs.

Stochastic systems are more complicated, because you have to think harder and imagine how you can tame their inherent randomness to produce output of consistent value. However, a bit of randomness can spice up an otherwise dull predictable app.

Static or dynamic

Does the content of the app change over time, or is it always the same?

Static content requires far less work. That's an understatement, even when bolded.

Users

We often assume users are people, but that need not be the case if we are abstract thinkers.

People

About people:

  • they usually like some feeling of control, whether illusory or not - a lot of design effort is devoted to this. Of course some people prefer to feel controlled, rather than in control in some circumstances.
  • often finicky and unreliable - human-facing apps need to be tolerant of the frailty of the human condition

Who are you designing for?

  • nobody at all
  • yourself
  • people like you
  • people not like you - be as specific as you think is useful
    • small children
    • elderly
    • disabled elderly with small children
    • immigrant disabled elderly with small children
    • immigrant disabled elderly single white female between 30-40 with income below $30k with small children living in Ossining, NY

What do they want?

What do you want from them?

What are they likely to let you do to them?

See an overview of user experience design, if interested.

Example app idea for people who are nothing like me:

Animals

  • not people at all... no need to be humane
  • generally uninterested in your promising new start-up idea
  • we tend to more clearly see animals' proclivity to respond to Pavlovian classical and operant conditioning, although this works equally well on humans
  • often have very different physical characteristics from one-another, so really not all one type of user

Example app ideas for animals:

Robots/automatons

Generally happy to be told what to do

Example app idea for robots:

Devices

Pedestrian devices

  • desktop - for those days when you need serious processing power
  • laptop - portable computers
  • tablet - good for use-cases that involve sitting on the toilet
  • mobile - slow, dumbed-down computer. most people's primary device

Laptop/desktop systems:

  • Microsoft Windows
  • Apple Mac OS X (based on UNIX)
  • Google Chrome OS (based on Linux)

Tablet/mobile systems

  • Windows mobile
  • Apple Mac iOS (based on UNIX)
  • Google Android (based on Linux)

More about operating systems.

Less-pedestrian devices

  • pacemakers
  • high-frequency trading systems
  • sneakers
  • whatever you make with arduino or raspberry pi

Lots of variation in operating system or platform when it comes to custom devices.

VR accessories

Since it's the hot thing

  • Oculus Rift
  • HTC Vive
  • PlayStation VR
  • Samsung Gear VR
  • Google Cardboard

See some miscellaneous notes on play-testing the commercially available VR systems for app development

Interaction design

Some common terms to use to appear smart when talking about interactiveness of a particular application:

  • Hot vs. cool media
  • Interaction as conversation
  • Doing, feeling, knowing
  • Cardinality of interaction
  • Symmetry
  • Beauty vs. ugliness
  • Lean forward vs. lean back experiences

See many other less catchy ways of quantifying and qualifying interaction design

Means of communication

How are your parts connected and communicating with each other? How is data transferred among multiple devices?

Electricity

Light

Wireless communication

The ability to communicate wirelessly results from several physical media that are essentially artifacts of the electromagnetic force!

  • radio
  • light
  • capacitance

And one wireless medium that is not a result of electromagnetism:

  • sound

More details on wireless communication

Magnetism

  • Hall effect sensors
  • Transcranial magnetic stimulation (TMS)

Radio waves

  • RFID
  • bluetooth
  • garage door openers
  • wifi
  • cell phones
  • FM & AM radio

Sound

Physical network topologies

How your devices are physically connected together can impact what kinds of virtual applications you can make.

Ring network

  • each device is only connected to the ones next to it
  • all devices together form a circular network
  • messages pass only from neighbor-to-neighbor
  • doesn't rely on a central hub
  • failure of any device may cause connectivity problems for the rest

Star network

  • all devices connected to a central hub
  • all inter-communication sent from one device through the hub to other devices
  • relies on a central hub
  • failure of the hub is a total failure of the network
  • lends itself to central control, censorship, and surveillance, of course

Tree network

  • A star network, where any given device may itself act as a hub for more second-tier devices
  • popularized by that thing called the Internet

Mesh network

  • ad-hoc, peer-to-peer connectivity
  • no clear organization
  • doesn't rely on a central hub
  • failure of any given device doesn't affect the others

Client-server architecture

The architecture made most popular by the web. See a little bit more of a written description of clients vs. servers.

Client

  • Any program that requests data
  • Writing such programs is often called client-side programming
  • Such programs are often called client software
  • The computer these run on are often called client computers

Thin clients

  • doesn’t do much more than request data from a server and displays what the server serves
  • any bothersome processing, data manipulation, or decision-making are left to the server to handle

Fat client

  • requests data from servers and displays information
  • but also does plenty of additional processing, data manipulation, and decision-making in its own right


Server

  • Any program that responds to requests
  • Writing such programs is often called server-side programming
  • Such a programs are often called server software
  • The computer these run on are often called server computers

Single server

  • a single computer that acts as a server
  • this computer can be stored in your bedroom, or a computer in a professional data center

  • it could be your phone or your Arduino
  • not usually easily scalable

Cloud

  • what appears to be a single server is actually a giant array of computers sharing resources
  • these computers may all be physically located together in one data center, or distributed around the world

  • provide for far more powerful processing
  • scalable

The web

What is the web?

  • a vision laid out by Vannevar Bush's in 1945
  • a vision implemented by Tim Berners Lee in 1989.
  • documents that link to each other
  • client/server communication using the HTTP or HTTPS protocols over the Internet

What is a web site

  • a document viewed in a web browser
  • usually refers to documents with mostly static content
  • a web site may be 'book-marked' on iOS or Android with an icon that loooks like a 'native app' icon, but it must 'run' in a web browser
  • code is interpreted by a web browser, which ultimately only understands a few types of code
    • HTML - used to represent what is on a web page
    • CSS - used to represent how that stuff looks
    • Javascript - used to indicate any changes to the content of a page that should result from interactions like clicking, hovering, typing, etc.

And a web app is what?

  • same as web site, but usually contains:
    • a data-driven web site
    • changes to the content of the site depending on context
    • usually interactive and dynamic in some way

Web apps will often additionally rely on the server responding differently to requests depending on context. Programming the server to analyze contexts and perform more sophisticated computing is termed the 'business logic' of an app. This is done by programming the server using a server-side programming languages, such as:

  • PHP
  • Python, specifically using the Django or Flask frameworks
  • Javascript, specifically using the Node.js using the Express.js framework
  • Ruby, specifically using the Rails framework
  • ASP, specifically using the .NET framework

Usually developers pick one of these languages and stick with it.

Native apps

Native app can access device media and data resources: camera, mic, accelerometer, GPS, etc. These are usually inaccessible to regular web sites.

Is a native app different from a web app?

  • possibly
  • possibly not

It's possible to wrap a web app into a container that allows it to be installed as a native app on a device. For example, see:

  • Adobe Cordova for mobile apps
    • You'll want to know HTML, CSS, and Javascript programming, and probably use a UI framework like Ionic
  • Electron for mobile apps
    • You'll want to know HTML, CSS, and Javascript programming

Of course, it is possible to build truly native apps that are not wrapped up web apps. To do this requires developing with the specific toolkit offered by each device operating system manufacturer:

Content

What is an app without content? Ultimately, there must be some data that the app allows a user to access or manipulate. Where does that data come from?

User-generated content (UGC)

UGC is a lazy producer's favorite type of content. Users enter it into the app themselves so you don't have to produce it.

  • privacy issues - are you insinuating that your users have a modicum of privacy while not actually giving them any?
  • moderation issues - protect against bots, trolls, and 'deplorables' (whatever it is you consider deplorable)
  • legal issues - HIPPA / COPA, liability
  • consistency and categorization issues

Editorial content

Created by the creators/editors of the app.

  • quality can be consistent - if staff are good
  • resource issues - cost, personnel
  • maintenance issues - you may find that regularly creating new content is more difficult than you imagined

Algorithmically-generated content

Content that a computer program has produced.

Aggregated content

Content scraped from 3rd-party publicly available sources, such as public agencies or web sites.

Content from APIs

Usually-proprietary content owned by 3rd parties that one must 'register' to gain limited access to. Examples:

Each API has their own system of authentication and their own instructions for how to use it. Most well-established APIs have documentation for how to connect to them using popular programming languages such as Python or Javascript.

Content types

Content can take a variety of types:

  • written text
  • images
  • 3d shapes and worlds
  • video
  • audio
  • geospatial

Each has its own storage, communications, and implementation issues.

  • file size
  • processor load
  • speed of download/upload

Data storage

Data that your application relies upon needs to be stored somewhere...can be stored

Client device

You can store data locally on a client device.

  • good for static content that does not need to be transmitted over a network more than once.

Server device

You can store data remotely on a server device.

  • good if data needs to be accessed by more than one client
  • can update data in one place and have it automatically updated by all clients

File system

When in doubt, store content in files on the hard drive

  • This can be fast and cheap for simple data
  • Good for data that does not lend itself especially well to other forms of data storage
  • Good for avoiding the complicated setup and management of a database

Databases

Use one if you have

  • a large quantity of data to be stored and manipulated.
  • a need for very fast, efficient data lookup and analysis
  • multiple clients accessing the same data

Databases can be either stored client-side or server-side, depending on your need

  • SQLite is a database that one often finds on the client-side
    • Mail programs
    • Web browsers
    • Python programs
  • Other database systems are always on the server-side because of their prohibitively large footprint:
    • traditional relational databases: MS SQL, MySQL, Oracle, etc.
    • MongoDB
  • Blockchain is always client-side because it is designed to allow transactions without a central authority (a.k.a. a server)

APIs

Store the data on a 3rd-party's database.

  • APIs are almost always server-side... usually on a server that you don't control

Databases

Databases provide fast and efficient storage, lookup, and manipulation of data.

  • see CRUD for most common needs in data storage systems

Three general types in the app world.

Relational

Data is stored like a spreadsheet.

  • any given piece of data is only stored in one place in the database
  • almost all relational databases require programmers to use the SQL language to issue commands to the database.

NoSQL

MongoDB is a popular NoSQL document-oriented database system.

  • any given piece of data could potentially be stored redundantly in many places in the database
  • MongoDB allows programmers to issue commands to the database using Javascript
  • includes geospatial features
  • easily scalable

Blockchain

Blockchain is an immutable decentralized database that is virtually tamper-proof while being transparent and open to the public.

  • most notorious as being the data storage system behind Bitcoin
  • Other decentralized storage systems, such as IPFS use blockchain technology
  • a whole generation of decentralized applications is currently being built to operate on the blockchain without central control.

Web apps

Web apps have been patterned ad nauseum. Don't let that deter you.

What's your stack, dude?

LAMP stack

  • Linux
    • the operating system on the server computer
    • should really be called GNU/Linux
  • Apache
    • the web server software that speaks HTTP to the web clients
  • MySQL
    • a popular and cheap relational database system\
    • familiar to old-timers
  • PHP
    • a server-side programming language that everyline loves to hate
    • dirt simple to set up
    • hackish syntax is off-putting to hardcore programmers
      • not as off-putting as Javascript is for newbies
    • more-or-less a 'roll-your own' sort of language
    • plenty of 'frameworks' and ready-made code to help you solve common problems

Some people replace the PHP with Python, since it also starts with a 'P' and looks more like other programming languages.

MEAN stack

  • MongoDB
    • a document-oriented NoSQL database system all the kids love
    • easily scalable
    • include geolocation abilities
    • data format in JSON
    • queries are written in Javascript
  • Express.js
    • an open-source server-side web application 'framework' for Node.js
    • a set of built-in functions that make creating the server-side code for a web application easier
    • written in Javascript
  • Angular.js
    • an open-source client-side Javascript library that includes data binding
    • automatically can update content based on new data received from a server
    • requires little custom code for basic applications
    • written in Javascript
  • Node.js
    • web server software that speaks HTTP to the web clients
    • nice routing features makes it good for creating custom APIs
    • writte in Javascript

MERN stack

Same as MEAN stack, but with Angular.js replaced by React.js

.NET stack

Also known as the WISC stack. This is Microsoft's Windows-centric development environment.

  • Windows
  • Internet Information Services (IIS) - Microsoft's web server software
  • SQL Server - Microsoft's main relational database product
  • C# - programming language, a variant of the Java programming language

Not trendy with the hip app kids. More popular for industrial/enterprise use, since it integrates seamlessly into Microsoft's other products.

Did you separate your view from your model, man?

MVC architecture

Many hold sacred a separation of the holistic unity of a web app into a holy trinity:

  • Model
    • a code representation of your data and what you can do with your data
  • View
    • the user interface code
  • Controller
    • the business logic code, i.e. the brains that decide what view to show the user, and what data from the model should be accessed or manipulated to use in every interaction

Limitless numbers of Stack Exchange and Reddit threads debating the merits of MVC vs competing architectures. For example.

Did you give your app SPA treatment?

Single Page Application (SPA)

  • the idea that the user interface code used to generate the View for any screen in an app should be downloaded once and only once by the client
  • the idea of a 'web page' is horribly out of date. these apps consist of only one page, but show multiple 'screens' or 'views'
  • Angular.js and React.js are often used to create these
  • Cordova (a popular wrapper that allows web apps to install as native apps on mobile devices) essentially requires your code be set up as SPA

Multi-page applications (no acronym for this)

  • what you're stuck with in web apps if you don't know Javascript
  • every page is like a separate program that starts from scratch
  • requires more data transfer between clients and servers
  • more data transfer means slower
  • requires all kinds of hacks and work-arounds to maintain 'state' within a web app

What web framework are you using?

Why reinvent the wheel when designing your client-side user interfaces? Most of the time, you're not doing anything new visually.

Bootstrap is a popular web design framework

  • comes with a lot of useful ready-made CSS code that looks more 'contemporary' than the browser defaults
  • comes with a grid system that makes it easy to create responsive apps
  • allows you to easily create common 'interactive' components on web pages like image carousels that would normally require Javascript programming skills
  • assumes you understand CSS code to some degree

JQuery is an extraordinarily popular client-side Javascript framework:

  • allows you to easily detect user-generated 'events' and change the page on the fly in response to them
    • mouse over
    • mouse out
    • keypress
  • comes with lots of useful functions in Javascript for modifying web page content on the fly
    • changing the styles of components of the page
    • loading new content from the server and showing it on the page (a.k.a. AJAX)
  • doesn't include any ready-made style or design elements

p5js is a client-side Javascript framework for animation and web graphics

  • allows you to create complicated Javascript animations and graphics by writing simple Processing code
  • popular among artists, educators, and people who are scared of code or just want to start messing around with no fuss.
  • low barrier to entry to an otherwise off-putting world of programming


Internet of Things device prototyping platforms

Apps are, of course, not restricted to laptops, tablets, and phones. As computer are now found everywhere, as you might expect their shape is changing and becoming more diverse.

Arduino

Every hackers favorite open source electronics prototyping platform.

Operating system:

  • No operating system
  • No file system
  • a tiny amount of Flash memory
  • cannot multitask

Interface:

  • must be connected to a 'real computer' to upload/download programs to it
    • typically via USB cable

Building apps:

  • typically programmed in Arduino (a variant of the C programming language)
  • code is written on a 'real computer' then downloaded to the Arduino
  • comes with a simple development environment for writing code
  • Javascript programmers may like the Johnny-Five platform to control an Arduino from Javascript code running on a laptop

Networking:

  • easily connected to the Internet or a local network
  • can connect via USB to a 'real computer'
  • can easily be hooked up to wifi with an adapter
  • almost always acts as a client.

Sensors / actuators:

  • can have an endless variety of sensors and actuators hooked up to it
  • typically achieved by buying additional read-made hardware circuit, although it's possible to roll your own circuits, of course.

Community

  • typically composed of hackers, activists, and DIYers
  • most likely to be building sensor based embedded devices and products

Raspberry Pi

A 'real computer' with closed-source hardware.

Operating system:

  • runs Linux
  • comes with a standard file system
  • has a microSD card slot
  • can multitask

Interface:

  • can be connected to a mouse and keyboard
  • or can be remotely logged-into

Building apps:

  • can be programmed in any language
  • can use any development environment that supports Linux
  • code can be written straight onto the Pi
    • Python is a commonly-used general-purpose language

Networking:

  • easily connected to the Internet or a local network
  • includes USB ethernet connectivity
  • can easily be hooked up to wifi with an adapter
  • can be set up to act as either a client or a server

Sensors / actuators:

  • can have an endless variety of sensors and actuators hooked up to it
  • lots of tutorials on building custom circuits, and a smaller-but-significant market for purchasing ready-made circuits
  • may require additional hardware interfaces

Community:

  • typically more experience computer programmer types
  • benefits from entire oeuvre of Linux documentation
  • more general purpose applications


What links here