Variants of Turing machines

There are many different ways we could define Turing machines without changing which languages are Turing-recognizable.
For example, Sipser defines the tape head moves as \(\{L, R\}\) instead of \(\{L, R, S\}\).
We summarize this by saying the definition is robust (not sensitive to changes), explaining its ability to model anything a computer can do.
Let’s consider how we might relieve the apparent limitation of a single sequential tape:
- Use multiple tapes (one for input and the other for scratch space).
- Use a two-way infinite tape (instead of one-way infinite).
- Use multiple tape heads on a single tape.
- Use a “2D tape” (i.e., an unbounded grid of cells in the 2D plane), a model of “scratch paper”.
- Use random-access memory (allow \(M\) read/write access to an auxiliary “address tape;” teleport the main tape head to addresses as they are read).
All of these appear more powerful than a single tape Turing machine, but aren’t.
How do we prove this? Simulate them on a standard TM!

Multitape Turing machines

A multitape Turing machine is an ordinary Turing machine but with \(k\) tapes.
- Each tape has its own independent read/write head.
- The input is loaded onto the first tape, and the other tapes start blank.
- Its transition function has the signature: \[ \delta: \fragment{(Q \setminus \{q_a, q_r\}) \times} \fragment{\Gamma^k} \to \fragment{Q} \fragment{\times \Gamma^k} \fragment{\times \{L, R, S\}^k} \]
  
  \[ \fragment{ \delta(q, a_1, \ldots, a_k) = (q', b_1, \ldots, b_k, m_1, \ldots, m_k) } \]
Theorem: For every multitape Turing machine \(M\) there is a single-tape Turing machine \(S\) such that \(L(M) = L(S)\). Furthermore if \(M\) is total, then \(S\) is total.
Proof idea: Construct \(S\) to simulate \(M\) on a given input.

In order to define \(S\), we have to describe how it:
1. Encodes any configuration \(C_M\) of \(M\) as a configuration \(C_S\) of \(S\).
2. Initially sets up its tape to simulate \(M\) (i.e., gets to the configuration encoding \(M\)’s initial configuration).
3. Moves from \(C_S\) to a new configuration representing \(C'_M\) when \(C_M \to C'_M\).

A configuration of \(C_M\) looks like: \[ C_M = (q, p_1, \ldots, p_k, w_1, \ldots, w_k) \qquad\qquad\qquad \]
Need to encode \(k\) tape head positions and contents!
Special marker \(\markedCharacter{ }\) encodes that a tape head is at a cell:
- Graphically: replace \(a \in \Gamma\) with \(\markedCharacter{a}\) (e.g., \(\markedCharacter{\string{x}}, \markedCharacter{\string{0}}, \markedCharacter{\#}\)).
- Formally: we can represent marked/unmarked characters as tuples in \(\Gamma \cross \{\circ, \bullet\}\) (e.g., \((\string{x}, \circ)\) and \((\string{x}, \bullet)\)).
To encode tape contents \(w_1\), \(w_2\), \(\ldots\), \(w_k\), we have a few options:
- Sipser’s approach: concatenate the tape contents with \(\#\) delimiters
  marking their starts and ends (assuming \(\# \notin \Gamma\)). \[ \# w_1 \# w_2 \# \cdots \# w_k \# \]
- Our approach: form “compound symbols” to represent the characters at the same cell of each tape.
  - Graphically: a single smushed-together symbol like \(\string{b} \markedCharacter{\string{0}} \string{x}\) to represent three tapes with respective symbols \(\string{b}, \string{0}, \string{x}\).
  - Formally: an element of \((\Gamma \cross \{\circ, \bullet\})^k\)

Given input \(x\), setup is easy in both variants.
- Sipser: \(\# \markedCharacter{x_1} x_2 \cdots x_n \# \markedCharacter{⎵} \# \cdots \# \markedCharacter{⎵} \#\).
  1. Write \(\# \markedCharacter{x_1}\).
  2. Copy the rest of the input from \(x\) with no markers.
  3. Write \(k - 1\) copies of \(\# \markedCharacter{⎵}\), followed by a final \(\#\).
- Ours: \(\markedCharacter{x_1}\markedCharacter{⎵}^{k - 1}\; x_2{⎵}^{k - 1} \; \cdots \; x_n{⎵}^{k - 1}\)
  - Write \(\markedCharacter{x_1}\markedCharacter{⎵}^{k - 1}\)
  - For each remaining character, write \(x_1⎵^{k - 1}\)
To simulate each transition:
- Scan through the tape until all \(k\) markers are found, and remember the symbols under them.
  This is a finite amount of information: \((a_1, \cdots, a_k) \in \Gamma^k\))
- \(M\)’s transition function can now be consulted \(\delta(q, a_1, \cdots, a_k)\).
- Do another pass through the tape the beginning, writing the appropriate symbols around each marked character to update the character and move the marker.

Move-right-or-reset: the tape head can only move right or reset to the leftmost position.
Two stacks
- But using just one stack reduces the power to a deterministic pushdown automaton, incapable of deciding even certain context-free languages.
One queue: a.k.a., “clockwise Turing machine”… why call it that?
Three counters: three nonnegative integers that we can increment, decrement (if not already 0), and check if they are 0
- But using one counter is equivalent to a unary single-stack machine, which can decide \(\setbuild{0^n 1^n}{n \in \mathbb{N}}\) but not \(\setbuild{x = \reverse{x}}{x \in \binary^*}\))
- Even two counters are Turing universal! (under a strange encoding of the integers)

Informally, an algorithm is a sequence of steps that can be followed to solve a problem.
Formally, we can define an algorithm as a procedure that can be implemented by a Turing machine.
However, after gaining some practice with Turing machines, we typically won’t specify the details of the machine.
Instead we can describe algorithms formally and precisely with a higher-level language (like Python).
We will argue that implementing a Python program on a Turing machine, while slower, won’t change its “complexity class.” (See HW3)

But programs can obviously operate on fancier objects than strings.
How can a Turing machine do this too?
For example, how can a TM read a graph?
- Node and edge lists: \(\encoding{G} = \text{(binary encoding of)}\qquad (\fragment{(1,2,3,4)}, \fragment{((1,2),(2,3),(3,1),(1,4))})\)
- Adjacency matrix: \[ \fragment{\begin{bmatrix} 0 & 1 & 1 & 1 \\ 1 & 0 & 1 & 0 \\ 1 & 1 & 0 & 0 \\ 1 & 0 & 0 & 0 \\ \end{bmatrix}} \fragment{\quad \Longrightarrow \quad \encoding{G} = } \fragment{0111101011001000} \hspace{15em} \]
  
  How can we determine the number of nodes?
  
  We use \(\encoding{O}\) to denote the encoding of an object \(O\) as a string in \(\binary^*\). We can encode multiple objects with \(\encoding{(O_1, O_2, \ldots, O_k)}\) or just \(\encoding{O_1, O_2, \ldots, O_k}\).

But programs can obviously operate on fancier objects than strings.
How can a Turing machine do this too?
For example, how can a TM read a graph?

The choice of encoding \(\encoding{\cdot}\) can matter a lot for practical reasons
- What is the size of the adjacency matrix representation for a graph with \(n\) nodes? \(n^2\)
- What about the node+edge list representation for a graph with \(n\) nodes and \(m\) edges?
  Proportional to \(n + m\). For sparse graphs \(m \ll n^2\)…
- This sort of difference is a big deal, e.g., in high-performance computing.
But for the sorts of performance differences we study in this course, we will show any “reasonable” encoding is good enough (won’t change whether an algorithm is polynomial time or not).