Selection and Navigation

Content Selection

Status of this Section

  • Semi-stable Draft

Description

The content selection interface, which includes the Home Location, is best described as a pair of columns, Column A and Column B.

The Home Location on the device will be Column A, which will contain a list
of Content Modules. Once the user presses the right arrow or the Select
button on one of the titles, he/she is taken to column B where a list of
audio segments that belong to the selected Content Module is displayed. If
the user pressed Play when at Column A, the device would simply start
playback at the beginning of the Content Module selected or, if the Content
Module was played in the past, the device would begin playing at the
location the user stopped the last time that module was heard.

All Content Modules are shown on Column A in the order they were downloaded
or saved to the device. The order is reverse chronological with most recent
at the top of the list.

When the user taps Home, he/she is taken to Column A, the same location the
user is placed at when the device is turned on. When the user holds Home,
he/she is told:

"Tap Play for previous Content Module, up or down arrow to choose from
categories."

Non-verbose: "Home to switch or arrows for categories"

The non-verbose version of voice prompts does not have to be extremely brief
since the user can always interrupt prompts by just pressing the desired
button.

Categories are simply tags that can be associated with Content Modules.
There are user and factory-defined tags and they are both treated and
displayed equally. They are shown when Home is held and up or down arrow
keys are pressed. Categories are displayed in the order they are created in
reverse chronological order from top to bottom just like Content Modules on
column A, i.e. at the Home Location. For this initial version we will only
have a Category called "All". Once category creation and assigning is
described and programmed, we wil continue to have "All" as an option but
will also be able to display by any of the other categories.

Question/Issue:

  • Note that this page has been substantially updated. (Cliff pasted in the text from a Fernando email.)
  • Note that I started a discussion topic for this section, as I didn't want to mark up this section. Please check and respond there.

Navigating within Content

General

This location is where users can listen to content modules and where they will spend most of their time. This location is used for listening to content modules designed for literacy learning and knowledge access.

Getting into this Location:

A user selects either a content module or a particular audio segment of a content module. See selection.

Getting out of this Location:

A user must take one of the following actions to exit this location:

  • hold the Help/Home button. See behavior below.
  • tap the Record button. See behavior below.
  • slide the power switch to off. The device resumes with the Home location after power is reapplied. However, the content module will be easy to get back to, because it will have been registered at the top of the "most recently played" list once the content module was selected. See power.

Initial Actions within this Location

Once user content module has been selected, the device will:

  • register the content module at the top of the "most recently played" list.
  • play the content module from its beginning (Package OnStart)

Button Behavior in this Location

Of the ten buttons of this device buttons, five of them have a fixed behavior and five of them can be reprogrammed by the content module's metadata.

Fixed Behavior Buttons

Play/Pause:
  • Tapping this button will alternatively play and pause the currently selected content module. When paused, tapping this button will cause the content to be played, from where it was last paused (whether by user action or by a content-embedded pause, such as may occur at a segment break). If the last paused point is at the end of the content, the playback will begin at the start of the content module (Package OnStart). If the device is woken from sleep mode, play will resume just as if there was no sleep mode (Last Point Played is remembered).
  • Holding this button at any time (during playback or when paused) will cause the device to jump to the speed control menu. From there, another tap will return to the playback or pause condition that existed before this action. See speed for more information.
Volume Up/Down:
  • Tapping and holding these have the same function at all times as in system module locations, as described in volume.
Home/Help
  • Tapping this button at any time (during playback or when paused) takes user into the Help location (during playback or when paused). This will also then allow the user to easily return to where they were (Last Point Played), to jump to the Home location, or to explore the help content, i.e. to learn about the device functionality or, if provided, about the content module. See help for more.
  • Holding this button will cause the device to go straight to the system module's Home location contexts.
Record
  • Tapping this button when the content is paused, which jumps to the Record location record. There is no response if the Record button is tapped during playback.
  • Holding this button is ignored. It is not equivalent to a tap. No action is taken.

Content Programmable Buttons

  • Content Programmable Buttons basically allow the author of a content module to create metadata that specifies an action that the device should take upon a user tap or hold of the button during a certain block of time within the content module. The most common action will be the goto action; however, the destination point must remain within the same content module (package). There are two categories of content programmable buttons:
    • Select
    • Arrow Buttons
Select
  • This button has no other function (within this location) than what the content module's metadata provides for it.
  • The functionality will normally be an audio hyperlink to learn more information about the current content being played.
  • It may also be used by the content module as an "enter" or "select" button, as it can be used within the system module.
  • It is expected that this button will usually be tapped, but the content module may also assign a function to holding the button.
  • For more information, see hyperlink in the functional spec or elements-onbutton and elements-goto for a lower level view.
Arrows Buttons
  • General
    • The arrow buttons described below each have default behavior that can be overridden/reprogrammed by the content module's metadata.
    • The buttons are intended to serve two primary purposes: 1) navigation within a content module, and 2) input for interactive content applications (e.g. multiple-choice questions).
    • These functions should not conflict with each other since a content module that uses an interactive feature may not need or want the user to navigate away from the interactive part of the module – and users can always exit a misbehaving content module (see "Getting Out of This Location" above).
    • The navigation functionality of these buttons is meant to work as follows: Left and Right buttons are smaller jumps back and forward; Up and Down are larger jumps back and forward.
    • If the device is set down on top of a page, one might imagine the Left and Right arrows represent back and forth across a line of text and the Up and Down arrows represent up and down the page, even though each pair may be used for jumps of arbitrary lengths.
    • The default behavior described below is meant to work best for information content, which is the only type of content that can be recorded by the devices.
    • Literacy content requires a more complex two-level hierarchy of points in text, such as word/line or word/page or line/page. Therefore, literacy content will only be created with the content authoring product, which will make it easy for authors to indicate points in each of these two levels and then reprogram the default behavior of all four arrow buttons.
    • However, information content can be useful with break marks between only one level of audio segments (such as between chapters or topics). A second manner of navigation uses relative time jumps, which doesn't require an extra level of segmentation to be designated by the content author.

IMPORTANT NOTE
The functionality of each arrow button described below must actually be specified in the record spec. It is listed here to provide an understanding of the resulting user experience, particularly of content that has no overridden defaults, such as content recorded on the devices. During playback, the device simply processes the metadata. The implementation of the defaults described below actually happens during the authoring stage, either by the content authoring product or by the device. In the case of a device recording, a very simple set of default metadata is attached to the recording, which will enable the default behavior described below. See the record spec for a definition of that metadata (still a TO-DO item as of March 23rd).

  • Left
    • Tapping this button jumps backwards 15 seconds behind the Last Point Played. If the device was in a playback state when the button was tapped, the device plays the content module at the new position; if the device was in a paused state when the button was tapped, the device remains paused. In both cases, a brief SAF is played to indicate the jump back. If there is less than 15 seconds to the beginning of the current audio segment ( block or file) the device jumps the remaining seconds back into the previous segment. If there is less than 15 seconds before the beginning of the entire content module (elements-package]), then the current position is moved to the beginning of the content module with the device, with the device in the same playback or paused state it was in before the button tap. In this beginning-of-module case, a second SAF is played to indicate the beginning of the content (this may only be a short tone, if not a spoken word/phrase….TBD).
    • Holding this button is ignored by default (unless defined by the content module). It is not equivalent to a tap. No action is taken.
  • Right
    • Tapping this button jumps forwards 60 seconds ahead of the Last Point Played. If the device was in a playback state when the button was tapped, the device plays the content module at the new position; if the device was in a paused state when the button was tapped, the device remains paused. In both cases, a brief SAF is played to indicate the jump back. If there is less than 60 seconds left in the current audio segment ( block or file) the device jumps the remaining seconds into the next segment. If there is less than 60 seconds left in the entire content module (elements-package]), then the current position is moved to the end of the content module with the device in a paused state, regardless of whether the device was paused or playing when the button was tapped. In this end-of-module case, a second SAF is played to indicate the end of the content.
    • Holding this button is ignored by default (unless defined by the content module). It is not equivalent to a tap. No action is taken.
  • Up
    • Tapping this button jumps back to the beginning of the current audio segment, unless the button is tapped within the first 1.0 second of the beginning of the audio segment, in which case it jumps back to the beginning of the previous audio segment within the same content module; unless the Last Point Played is in the first segment of the content module, in which case it still jumps to the beginning of its segment. If the device was in a playback state when the button was tapped, the device plays the content module at the new position; if the device was in a paused state when the button was tapped, the device remains paused. In both cases, a brief SAF is played to indicate the segment jump back. If the new position is the beginning of the content module, a second SAF is played to indicate the beginning of the content (this may only be a short tone, if not a spoken word/phrase….TBD).
    • Holding this button moves the current position to the beginning of the content module, just as if the same button was tapped more times than there are previous content segments.
  • Down
    • Tapping this button jumps forward to the beginning of the next audio segment or to the end of the content module if there is no next audio segment. If the device was in a playback state and not in the last audio segment of the content module when the button was tapped, the device plays the content module at the new position; otherwise, the device remains paused (or becomes paused in the end-of-module case). In all cases, a brief SAF is played to indicate the segment jump forward. If the new position is the end of the content module, a second SAF is played to indicate the end of the content (this may only be a short tone, if not a spoken word/phrase….TBD).
    • Holding this button moves the current position to the end of the content module, just as if the same button was tapped more times than there are following content segments.

Hyperlink

The hyperlink should ideally be indicated by a background sound that begins and ends with the word or words that represent the hyperlink. If that level of precision is difficult to achieve in the content development platform, the indicator will still work even if the sound finishes towards the beginning of a subsequent word in the recording.

The sound will be 70Hz, but this frequency is subject to change once feedback is in from beta testers. It is likely that it might have to be slightly higher in frequency so that it can be more easily heard. The key balance is to have a sound that is noticeable but not disruptive.

The user will access or activate the hyperlink by tapping the hyperlink key during or after the sound is heard. In other words, the hyperlink key will activate any current or the latest hyperlink on any given audio text.

Since content that is played after a hyperlink is activated can have hyperlinks of its own they will by default contain at least one hyperlink that brings the user back to the previous segment. The "Go back" hyperlink that must be on every recorded segment by default will function very much like the back function on web browsers. It will bring the user back to the hyperlink that was clicked on, on the previous segment. If the need to save excessive writes to flash memory exists, then the "go back" hyperlink will bring user to the beginning of the segment he/she came from.

  • [Cliff] I think we can use the "Left" button for this "go back" function when the button is tapped near the beginning of the hyperlinked content, see nav.

The main body of the recorded document, i.e. content module, will not have a "go back" hyperlink.

The content below should be moved to help and is triggered by holding the Home button (see buttons)

Pressing down and holding the hyperlink button for two seconds or more will play the message:
"Help Mode activated. Press any key to learn its function or press this key again for two seconds to deactivate help mode."
and will activate help mode. Pressing and holding it in the same fassion will play the message:

"Help mode deactivated. All keys will function normally now."
and take the user out of help mode and back where he/she was.

While in help mode pressing the hyperlink key for less than two seconds, i.e. a normal pressing, will cause the device to play the help description for that key. The same will happen for every other key on the device.

Pressing any key other than the hyperlink key for two seconds while in help mode, will cause the extended help segment to be played for the corresponding key. This will also be a recording explaining the function of the key, but in greater detail.

Content Embedded Pauses

embeddedpause
See also some crazy notes at old-nav

Speed Change

The Talking Book device offers playback speed control in order to facilitate learning at the student's own pace. When the student is learning a new word and sound, it's important that they can play the audio at a slower speed so they can hear it better, and perhaps read the corresponding written words. For navigational ease, it's desirable to allow students to increase playback speed, either as a fast forwarding mechanism, or as a way to refresh material in a short period of time.

The device will allow the following play back speed, listed in the order of slowest to fastest:
(Slower playback speed is crucial for learning, and should be prioritized above faster speed, which is more of a navigational helper and not essential to the mission of the device. If cuts need to be made, cut the faster speeds first)

  • 0.5x, or half the normal speed, at constant pitch, so that the student can learn the pronunciation and sound without having to adjust to pitch changes.
  • 0.65x normal speed, at constant pitch is required.
  • 0.8x normal speed, at constant pitch is required
  • 1x or normal playback speed
  • 1.5x normal speed, or 50% faster than normal. Constant pitch is preferred, and should be available as part of the chip set's feature.
  • 2.0x normal speed. Constant pitch is preferred, and should be available as part of the chip set's feature.

The slower speeds are determined from [citation needed] academic studies. The faster speeds are based on digital voice recorder specs only.

We examine all of buttons on the device and found there are no "free" buttons available to put the playback speed buttons:

  • Volume Up/Down should be dedicated.
  • MORE button is used for hyperlink and may be needed at any point in program playback
  • BACK/FORWARD are for jumping back 15 secs or fast forward 30 secs during playback
  • UP/DOWN are used for module/chapter navigation during playback

Thus it's necessary to somehow overloading of existing buttons to accomplish playback speed changes. We examine a number of button schemes:

  • Rapid double-tap of the button: The double-tap speed is hard to gauge and hard to learn. It is probably an unfamiliar concept for non-computer users, since it was introduced via the mouse. Rejected.
  • Tapping of two buttons:
    • Two variations were discussed, either the simultaneous pressing of two buttons, or a key-modifer approach where one button is held down while another is pushed.
    • Pros: There are parallels of simultaneous button pushes in the mechanical world: Some cassette recorders require pushing down of REC+PLAY to start recording. Typewriter's SHIFT key is a similar concept of key modifier and may be familiar to some users. It is easier to demonstrate to new users, without worrying about the timing element which which can be tricky to learn and get right.
    • Con: The two button maneuver requires a bit more dexterity, and is a new interaction that is not used elsewhere on the Talking Book.
  • Holding down of a button for >1 sec:
    • a bit of background: An important point is that a button will only appear to be "tapped" if it is depressed _and_ released within a short time, 1sec or less. The exact duration is TBD.
    • When the button is held (> 1 sec) the device can trigger the HOLD action without ever having triggered the "tap" action.
    • Pro: Since any button held longer than 1 sec is considered a HOLD, there is no tricky timing involved. Since we believe a button modifier scheme such as SHIFT or CTRL is likely the right behavior for any two-button gestures, which will involve a HOLD behavior to begin with, we think we should utilize HOLD by itself as a first step.

Design:

  • When a content module is selected, and PLAY is hit, it should always start at 1.0x speed
  • When a paused content module is resumed from a pause state, the playback speed should be the last used playback speed. For example, if I newly selected a module, selected playback at 0.65x, and paused it, but hours later resumed playback, the resumed playback should be still at 0.65x.
  • [FUTURE] It maybe useful to insert the speed back into the metadata for the module, so whenever the next time a given module is played, the same selected speed is used. This will involve, however, some consideration about reseting speeds when a module is copied to another device, etc.
  • Whenever the playback speed is changed, a corresponding SAF is played to tell the user the speed at which audio will play. They are:
    • 0.5x has the SAF as "Slowest Speed".
    • 0.65x has the SAF as "Much Slower Speed"
    • 0.8x has the SAF as "Slow Speed"
    • 1.0x has the SAF as "Normal Speed"
    • 1.5x has the SAF as "Faster Speed"
    • 2.0x has the SAF as "Fastest Speed"
  • When the PLAY button is held down for > 1sec
    • Audio playback is paused, the location is remembered
    • An SAF announces "Audio is paused. Press UP to speed up audio playback, press DOWN to slow down. Press Play again to continue."
    • The user can then tap UP or DOWN buttons to change the playback speed. Each tap will change the playback speed up or down one notch. The SAF corresponding to the newly selected playback speed is announced, i.e. "Slowest" or "Faster", etc.
    • NOTE: The SAF for the speed selected should be played back at the appropriate speed. For example, the SAF for "Slowest" should be played back at 0.5x speed; "Normal" at 1x; "Faster" at 1.5x, and so on. This is to help the user experience how the selected speed will sound.
    • The user can continue to adjust the playback speed up or down until the desired speed is attained.
    • If the playback speed is already at the slowest speed, pushing DOWN will not change the speed further. The SAF for "Slowest" is still played however. The same is true when the speed selected is already the fastest and the user tap UP.
    • When the user is satisfied with the speed selected, he taps PLAY again to continue playback at the selected speed. Audio resumes where it was left off.