Thursday, November 29, 2018

Sonification and Simple Auditory Displays with Arduino

Sonification and Simple Auditory Displays with Arduino

Stand-alone Arduino projects built by sighted people often use LEDs and LCDs as output methods. The color and placement of an LED might indicate a state or activity, while quantitative data like numbers or a simple graph are often presented on an LCD.

Of course, neither of these methods works well for blind makers or users. Instead, we prefer non-visual interface options such as sound, speech, or haptic feedback. I recently posted an article on using servos for quantitative haptic feedback (servometers) , so now I'd like to take a step back to introduce some simple accessible Arduino output techniques using sound. Other output methods such as vibration and text-to-speech are also extremely useful and articles on those topics are forthcoming.

What is Sonification and Auditory Display

An auditory display is exactly what it sounds like – the use of sound to represent information. There are lots of ways of doing this including alarms, alerts, transformation of data into sound, etc. Sonification is a type of auditory display that uses sound (not speech) to represent quantitative information. Usually sonification uses a specific set of sounds and varies the qualities of the sound depending on the information being represented. Since it is really easy to manipulate the pitch of a tone, many sonification techniques have used pitch to represent information. In other words, by associating a particular variable with the pitch of an output sound, we can make an auditory display that represents what is happening with that variable. If the pitch goes up it lets us know that the value of the variable increased. If the pitch goes down then we know the variable decreased. Of course, pitch is not the only sound parameter that can be used, but it's extremely easy to implement and control with the Arduino. It's also relatable for most users and it's classic, so that's what I'll be discussing in this article. Other parameters for sonification include timbre characteristics, pulse rate, and many others. Sound level (volume) is generally not recommended as a display parameter because users usually like to control that. You can learn much more about sonification and Auditory Displays from Wikipedia , or with a little additional Googling.

Making Sound With Arduino

You can buy a number of fancy shields for playing sounds and music with an Arduino, but we are going to confine the discussion to making sound with the classic Arduino Uno (available from Amazon) without any additional hardware other than an unamplified 8Ω speaker (also available from Amazon) . You can also use a piezoelectric buzzer, but the resulting sound quality is rather disappointing. Although buzzers are certainly loud enough (generally too loud), the quality of the sound they produce can be rather hard on the ears. They are also not reliable when reproducing pitch. I strongly prefer using a small, 8Ω speaker, sometimes with a 100Ω resistor in series with it if you are worried about drawing too much current.

There are two simple functions that are generally used to produce sound from an Arduino. These are analogWrite() and tone() . Both of these functions use Pulse Width Modulation (PWM), so in trying out the following examples, be sure that one wire of your speaker is connected to a digital pin that can support PWM (i.e., 3, 5, 6, 9, 10, or 11 on an Arduino Uno). The other speaker wire should be connected to GROUND.

Simple Beep Sonification with analogWrite()

The analogWrite() function lets you use a speaker to make audible beeps of various durations and intensities. This is really useful for providing simple auditory feedback for lots of projects. For example, you can use one, two, or three quick beeps to let the user distinguish between three different states or modes. You can make a long beep to mean one thing and a short beep to mean another. Beep rate can be used as an auditory display for a smoothly varying value with the time between beeps corresponding to the value (or its inverse) being represented like a Geiger counter.

The analogWrite() function is part of the core Arduino library. When used it tells the processor to start alternating the pin state between low (0V) and high (5V) at about 1000 times per second depending on which pin you are using. It continues to alternate low and high at this rate until you tell the pin to do something else. This alternation between low and high becomes a tone when you connect a speaker (or piezoelectric buzzer) to the output pin.

analogWrite() takes two arguments: the pin number and a number between 0 and 255. The pin number obviously tells the Arduino which pin to use, but the second argument tells the Arduino how much time to spend in the high state and how much time to spend in the low state. The percentage of time that the pin spends in the high state is called the Duty Cycle. A value of 0 for this second argument corresponds to a duty cycle of 0% (none of the time is spent in the high state), while a value of 255 tells the pin to spend all of its time in the high state (100% duty cycle). In the middle of the range at 128, the pin spends equal amounts of time in the high state and the low state (a 50% duty cycle). The cool thing about this is that, for sound, it gives you digital control of how loud the sound is. Low values for the duty cycle are quieter and high values are louder. Above approximately 10% duty cycle the loudness no longer increases, but using values below about 30 lets you adjust the sound level quite nicely. This is particularly useful if you are using a piezoelectric buzzer instead of a speaker because they can be extremely loud and obnoxious at full volume.

Sonification Example with analogWrite()

The following sketch illustrates how to use analogWrite to make a beep whose volume can be controlled while the auditory display demo uses the rate of the beeps to sonify the sensor reading.

 * This sketch demonstrates how to use analogWrite() to make audio beeps,
 * how to implement a simple, beep-rate auditory display,
 * and how to adjust sound volume.
 * For details on analogWrite(), see:    
 * Note that pins 5-6 on Uno and similar boards will give you a tone close to 1KHz
 * while other PWM pins will give you something closer to 500 Hz.
 * Wiring:
 *   Speaker -- 1 lead to pin 9, the other to ground.
 *   10K volume pot -- 1 end to 5V, 1 end to ground, arm to A0.
 *   10K sensor pot -- 1 end to 5V, 1 end to ground, arm to A1.
 * By Josh Miele for the Blind Arduino Project, 09/17/2018

//pin assignments
const byte volPotPin = 0;   //pot to control volume
const byte sensorPotPin = 1;   //pot to simulate a sensor reading
const byte speakerPin = 9;  //attach an 8 ohm speaker to this PWM pin
//Sound parameters
int maxVol = 20;   //maximum volume
int beepDur = 50;  //Beep duration in milliseconds 

//Delay timing parameters
int minDelay = 25;   //Minimum delay in milliseconds
int maxDelay = 1000-beepDur;  //max delay corresponds to 1 beep per second

void setup() {
  pinMode(speakerPin, OUTPUT);
}   //end of setup

void loop() {
  //read from volPot to set the duty cycle of the beep
  //use map to get the duty cycle param into the right range.
  int dutyCycle = map(analogRead(volPotPin), 0, 1023, 0, maxVol);
  delay(1);   //for stability)

  //read sensor pot to simulate a varying parameter
  //This could be any dynamic value from sensors, serial, or cloud.
  //Use map to scale the input to the right range of delays
  //Note that this should be exponential, but...
  int delayTime = map(analogRead(sensorPotPin), 0, 1023, minDelay, maxDelay);
  //Make the beep
  analogWrite(speakerPin, dutyCycle);   //Turn on the sound
  delay(beepDur);  //Wait for the length of a beep
  digitalWrite(speakerPin, LOW);   //Turn off the sound
  delay(delayTime);  //between minDelay and maxDelay
}   //end of main loop
Notes About Designing Auditory Displays with analogWrite()

If you are planning to implement a simple auditory display using beeps, here are a few pointers. Generally speaking people are good at distinguishing between long beeps and short beeps (think of Morse Code), but are not so great at estimating beep duration, so it's not recommended to use beep duration as a display parameter. On the other hand, people are pretty good at hearing very slight differences in timing between beeps(e.g., tempo). This means that varying the delay between beeps makes a pretty good auditory display. As mentioned above, using sound level as a display parameter is discouraged for a variety of reasons. The sketch above has a couple of flaws, most notably, the use of delay prevents the auditory display from responding immediately to changes in the sensor value. This can definitely be dealt with, but it would make the code much more complicated, so it is left as an exercise to the student. The other issue with the sketch is that we vary the delay linearly with the sensor input, but really it should be exponential in order to make the display more usable. More about that in the next section.

Sonification with Tone()

analogWrite() is great for just making an unspecified beep, but it provides no control over the pitch of the sound. For that we need to use tone() . This built-in function allows us to use the same speaker or piezoelectric buzzer to play a tone at a particular frequency, so we can now use pitch as a quantitative auditory display parameter.

The tone() function accepts three arguments: pin (the pin the speaker is attached to), frequency (in Hertz), and duration (in milliseconds). If you omit the duration parameter, the tone plays until you actively turn it off with another function like noTone() or digitalWrite(). For example, if you have your speaker attached to digital pin 9 and you want to play a middle A for 1 second, the command would look like this:

tone(9, 440, 1000);

Some Notes on Frequency, Pitch, and Perception

It's important to understand the difference between frequency and pitch. If all you want is a tone that gets higher and lower as you vary a parameter, then it's fine to simply use linear frequency. Increasing frequency makes a tone sound higher, and decreasing frequency makes it sound lower, but this simple approach leaves most of the pitch change at the bottom of the range. It is as if you have a speedometer that gives you great sensitivity at slow speeds, but at higher speeds it gets less and less accurate. To make the most use of the auditory display, we should have the pitch change equally distributed over the range of frequencies we want to use.

Frequency is the number of cycles per second of the sound. It's measured in hertz (Hz). 1Hz is one cycle per second. The audible range of frequencies for humans is approximately 20Hz to 20,000Hz, with most people losing some range at the top with age.

Another important thing to know is that we perceive pitch in cycles called octaves. The notes of any scale will repeat as we play up through the scale to the next octave, and the next, and the next. Every note has a fundamental frequency associated with it (e.g., middle A if 440Hz). Doubling that frequency gives you the fundamental frequency of that same note in the octave above, while half of that frequency gives you the fundamental frequency of that same note in the octave below. For example, 220, 440, and 880 all have the pitch of A, and are in three consecutive octaves, but the frequency difference between them is far from equal.

Putting it in mathematical terms, there is an exponential (or logarithmic) relationship between frequency and pitch.

This matters because humans are extremely used to hearing tones as pitches. Unless the exponential relationship is used, differences at higher frequencies will sound much smaller than differences at lower frequencies. For example, imagine we hear a tone at 100Hz. If we increase that frequency by 100Hz up to 200Hz, it has gone through an entire octave of change. On the other hand, if we start with a tone of 800Hz and increase the frequency by the same 100Hz up to 900Hz, the pitch only increases by a little bit – much less than an octave. This means that the same amount of increase in frequency sounds much smaller when it's a higher frequency than when it is a low frequency. Using the exponential relationship fixes this problem.

Putting it a different way, and using some musical terminology, if you want the pitch of your sound to change by a fixed interval for a fixed input, you need to include the exponential factor.

In fact, it turns out that most of our perceptions follow this exponential relationship, including our ability to hear tempo changes. This means that in our previous example we really should have used an exponential function instead of map to control the delay. This is psychology's well-known Webber Law which basically says that as stimuli get bigger (e.g., intensity, frequency, duration, etc.), you need a correspondingly larger difference in that stimulus in order to notice it as much.

Pitch Sonification Example with tone()

In the following example, tone provides output for a value being read from an analog input. In this case the input is connected to a potentiometer for demonstration purposes, but we could just as easily use a sensor of some kind, or be reading a value from a serial or cloud-based device. The point is that we're using tone() to represent a dynamically varying, real-time input. Note that this is not that different from the PitchFollower example sketch included with the Arduino IDE. Unlike that example, though, this version offers two different pitch Sonification methods – simple frequency variation, and a more sophisticated (better) linear variation of pitch with input value. This sketch also provides an easy way of turning off the sound to preserve your sanity.

 * Simple Sonification 
 * This sketch demonstrates the basics of using tone() for sonification.
 * It includes a quick-and-dirty qualitative approach using the value of 
 * analogRead() as the frequency. It also illustrates a more 
 * sophisticated approach, using the exponential function to keep 
 * display pitch linear with input value.
 * Wiring:
 * 8 ohm Speaker -- one lead goes to pin 9, the other goes to ground
 * 10K linear Pot -- One end goes to 5v, the other end goes to ground. 
 *     The arm of the pot goes to A0.
 * Josh Miele for the Blind Arduino Project -- 10/24/2018

//Specify pin assignments
const byte speakerPin = 9;   //Needs to be a PWM pin
const byte potPin = 0;   //any analog pin will do

//Set up some variables for the exponential sonification
float minVal = 10;   //Minimum sonified value
float maxVal = 1023;   //Maximum value from analogRead()
float octaves = 3.0;    //how many octaves in the display range?
int baseFrq = 100;     //Lowest display frequency

void setup() {
  pinMode(speakerPin, OUTPUT);
}   //end of setup routine

void loop() {
  //Read the pot and play the tone.
  int potVal = analogRead(potPin);  //we get vals between 0 and 1023

  //if the sensor value is above 10, then we play a tone
  if (potVal > minVal) {
    /* There are two different sonification versions below.
     *  Only one should be used at a time, but try experimenting 
     *  with each by uncommenting it and commenting the other.
     *  The first version simpley uses the output of analogRead()
     *     as the frequency value. This has most pitch change in the 
     *     bottom of the range. 
     * The second version uses an exponential function to 
     *     output equal pitch change for equal input change
     *     throughout the range of analogRead() values.
     *     Try also experimenting with different values of octaves and baseFrq
    tone(speakerPin, potVal);   //Frq is exactly equal to sensor input
//  tone(speakerPin, baseFrq*pow(2, (octaves*(potVal-minVal)/(maxVal-minVal)) ) );  //exponential sonification
  } else {
    //turn off the sound when the pot is all the way down.
    noTone(speakerPin);   //To preserve your sanity!
  delay(10);        //Just for stability...
}   //end of main loop

Applying Simple Sonification

There are lots of ways to use tone() and analogWrite() to represent data. After reading this article you can probably imagine how we have used similar approaches to build auditory continuity testers, obstacle detectors, carpenters levels, volt meters, and many other devices. You can combine these simple Sonification techniques to create pitch displays that (for example) pulse in a particular region of the display to indicate critical values. Other displays use tones that are audible only when the input value changes, but are silent otherwise. The simple techniques described above can be used as the building blocks to form some extremely sophisticated auditory display tools. Please let us know how you use them!

Friday, September 14, 2018

Products and Purchases

Occasionally we provide links in the Blind Arduino Blog to products we think might be useful for blind makers. We thought that we might be able to use these links to help support our efforts, so we've gone ahead and become an Amazon Affiliate which gives us a small percentage from your purchase when you buy stuff from Amazon through the links on the blog. We're not going to go crazy with this or anything, but we did want our readers to understand that we're doing this, and that when you use the product links in our posts to buy things through Amazon, you're helping us continue to do what we do. Thanks!

Thursday, September 13, 2018

ServoMeters -- Interactive Tactile Gauges with Servos

The Basics

Makers loves servos and, perhaps not surprisingly, some of my favorite servo applications are for building accessible haptic displays such as interactive tactile dials, gauges, and meters.

Servo motors are ubiquitous in hobby robotics. They are awesome little motors that incorporate a feedback system, making them extremely easy to use with Arduino and other microcontrollers. A few decades ago they were quite expensive, having most applications in aerospace or other fancy fields, but with the growth of hobby robotics and the maker movement, to say nothing of the popularity of RC vehicles, the size and price of servos have come down significantly at the same time as the reliability has gone up. Today you can buy little servos for well under $10 each and they are well supported with a Servo Library in the Arduino environment.

Servos come in two basic flavors: those with a range of motion of 0-180° and those that can rotate freely through 360°. Both kinds tend to be small rectangular boxes with a motor shaft sticking out of one side capable of mating with a variety of wheels, gears, arms, and other shapes designed to apply torque in one way or another. 180° servos have a built-in position sensor that lets you set the motor shaft to a specific orientation. Continuous rotation 360° servos have similar sensors, but for motor speed and direction instead of position. We will return to the topic of 360° servos presently. There are many other kinds of specialized servos, but for now we will stick to discussing these two types.

The 180° servos are capable of rotating their shafts through a half rotation and back again like a windshield wiper. More interestingly, you can tell the 180° servo to go to any position between 0° and 180°, and it will faithfully turn the shaft to that orientation and keep it there, even if you physically try to turn it away from that orientation. It's able to do this because it has a built-in sensor (usually a potentiometer) that lets it know what position it's in – if you try to force it away from where it's supposed to be, it tries to move back. This means it can reliably be placed at a specific angle and be counted upon to stay there.

Which Wires Are Which?

Before continuing with the discussion of using servos for accessibility we need to make the servos themselves accessible. Most servos have a color-coded, 3-conductor ribbon cable with a 3-pin female connector on the end. The colors are great if you can see, but if you can't, you'll need to know what's what with the wires.

Place your servo on a flat surface in front of you with the shaft pointing up and the ribbon cable toward you, making sure that the cable lies flat and is not twisted. In this configuration, the wires in the cable are FROUND, POWER, and SIGNAL from left to right for most servos. Check your product documentation if you are using an unusual servo.

Analog Gauges and Why They're Cool

Analog gauges have many distinct advantages over digital or text-to-speech for quantitative output. They are dynamic so readings can be constant rather than sampled at longer intervals. For example, a gauge that wiggles every time a wire is touched lets you know immediately that there's a loose connection, whereas an instrument with only a digital display might not catch this. A quick glance at a gauge lets you know if you are within an acceptable range much more efficiently than a digital readout, and a slightly longer look at the gauge gives you a reasonably accurate quantitative reading. All of these benefits apply to tactile versions of gauges as well.

Blind people have a long history of removing the glass from gauges in order to be able to read the position of the needle by touch. This worked great for analog alarm clocks which were designed to be thrown around a bit, but in most cases the glass is there for a reason: the needle is not designed to be touched. A delicate needle can move while being touched leading to a false reading. Worse, touching the needle could cause a delicate measurement instrument to permanently lose its calibration, leading to a long future of incorrect readings.

Meet the ServoMeter

The 180° servo is an ideal tool for building reliable and inexpensive gauges and meters designed to be touched – a category of displays I have dubbed ServoMeters. The robustness and pushback of the 180° servo is perfect for building tactile gauges, and the addition of tactile markings or braille numbers around the dial makes for an inexpensive, simple, and powerfully quantitative tool.

Servos are also much less expensive than TTS boards, putting tactile displays within easy reach of even the most financially strapped blind maker.

The most basic example of a servometer can be found right in the libraries directory of your Arduino development environment under Servo > Examples > Knob. The Knob.INO sketch demonstrates how to set the position of a 180° servo using a 10K potentiometer. From this example it is only a short hop of the imagination to see how you can use a servo to represent almost any quantitative Arduino output. The following is a modified version of the Knob.INO sketch I have used for teaching about servometers – a fundamental concept of accessible hardware design.

 Controlling a servo position using a potentiometer (variable resistor) 
 by Michal Rinott  
 Modified by Josh Miele for the Blind Arduino Project, 05/11/2018

#include <Servo.h>   //tells the compiler to use functions contained in the servo library

Servo myservo;  // create servo object to control a servo

int potPin = 0;  // analog pin used to connect the potentiometer
int servoPin = 5;  //Servo attached to digital pin 5
int val;    // variable to read the value from the analog pin

void setup()
  myservo.attach(servoPin);  // attaches the servo on pin 5 to the servo object
}  //end of setup

void loop() 
  val = analogRead(potPin);            // reads the value of the potentiometer (value between 0 and 1023) 
  val = map(val, 0, 1023, 0, 180);     // scale it to use it with the servo (value between 0 and 180) 
  myservo.write(val);                  // sets the servo position according to the scaled value 
  delay(15);                           // waits 15 milliseconds to stabilize the servo
} //end of main loop

In reality, many inexpensive 180° servos are a little unreliable at the extremes. It's generally best to limit your 180° servometer's range of motion to 160° (i.e. not using the 0-10° and 170-180° ends of the semicircle) as we have done in the sketch above. This can make placement of tactile markings something of a squeeze unless you have a rather long pointer attached to your servometer. For this reason it's often nice to have a gauge with more rotation than 180°. In fact, it's nice to be able to aim a pointer in any direction you want, giving you maximum flexibility with your servometer. Imagine, for example, trying to build a tactile digital compass.

The Continuous Rotation ServoMeter

Arduino communicates with continuous rotation 360 servos using the same servo object as the 180° servos. The Servo.write method accepts a value between 0 and 180, making it super easy to implement the 180° servometer. However, it is not so simple for 360° servos. Continuous rotation servos respond to the 0-180 input by rotating the servo shaft at a particular speed and direction with an input of 90 being no rotation, values slightly larger than 90 rotating the servo shaft slowly clockwise, and values slightly less than 90 rotating the shaft slowly counterclockwise. As values approach 0 or 180 the speed of rotation increases in a counterclockwise or clockwise direction respectively. This is cool, but it makes it pretty hard to rotate the shaft to a particular orientation and keep it there as is needed for a servometer. Luckily, you can buy a special 360° servo with a position feedback sensor built right into the unit. Parallax 900-00360 Feedback 360° High Speed Continuous Rotation

The product I have been experimenting with is the Parallax Feedback 360° High Speed Servo. It includes a Hall Effect sensor that provides reliable feedback regarding the orientation of the servo shaft. It requires a little knowledge of Arduino hardware interrupts and some mental-rotation gymnastics to implement something like the Knob sketch above, but it's not that complicated. The following short sections explain a little about how it works, but if you are impatient, feel free to skip it, download the knobFBS sketch, and start experimenting.

Sensor Feedback

The orientation of the servo shaft is measured not with a pot as is often done with 180° servos, but with a magnetic sensor that uses something called the Hall Effect. Feel free to study up on this cool electromagnetic phenomenon, but you don't need to understand how it works. What you do need to know is that, when the servo is hooked up to power and ground, the sensor sends almost a thousand pulses per second to the Arduino, and that the duration of each pulse is proportional to the angle of rotation. When the shaft is oriented to an arbitrary zero point, the pulses are at a minimum and are extremely short (about 24 microseconds). As the shaft rotates clockwise the duration of each pulse increases until the shaft has rotated almost all the way around to the starting point where the pulses will again become extremely short. Just before returning to zero, the pulses are much longer and are at a maximum (about 1076 microseconds). By measuring the duration of each pulse with the Arduino, and with knowledge of the exact durations of the shortest and longest possible pulses, we can calculate the current orientation of the shaft.

Being able to measure the orientation of the shaft makes it possible to use the servo speed and direction control to automate the movement of the servo shaft to any desired position – exactly what we need for a servometer. It also provides the possibility of using the feedback servo not just to display data, but also as a knob that can be adjusted by the user and read by the Arduino – a nice option for interactive accessible data display and input.

Measuring Minimum and Maximum Pulse duration

The best way to measure short pulses with an Arduino is by using external interrupts. Please feel free to explore this fascinating topic on your own, but for this project you just need to know that it's possible to use interrupts to instantly trigger desired actions through activity on certain digital pins. These actions take place regardless of what the main loop is currently doing, so interrupts are really great for catching input events that might otherwise be missed by a slower loop routine.

The following basicFBS.INO sketch allows you to measure the minimum and maximum pulse durations for your individual unit. It also provides a simple Sonification scheme for the servo's position sensor so you know it's working – as the servo turns clockwise the tone rises to about 1KHz and then jumps back down. Use a serial monitor and a screen reader to read the minimum and maximum pulse widths for use in the next section. I've tried to explain all the specific technical details in the code and comments, so take a careful read through this sketch to understand what's happening.

* This sketch shows the basics of how to read from a 360Deg servo with feedback.
* Apot controls the direction and speed of the servo and a speaker sonifies the servo position.
* The serial monitor is used to read exact max and min values for the feedback pulse duration.
* For use with the parallax feedback 360° high speed servo
* Feedback provided via Hall Effect sensor
* Duty cycle of sensor corresponds to position with a min of 2.9% frq approx 910 Hz
* (my unit min = 24 microS, max = 1076 microS)  
* Wiring:
* Servo has a ribbon of 3 conductors and a second single wire, all with female connectors.
* With the servo shaft pointing up and the wires coming straight from the unit toward you the ribbon connections
* from left to right are GROUND, POWER, AND CONTROL. The single conductor is the feedback SENSOR.
* Connect CONTROL to D9 and the SENSOR to D2, with GROUND  and POWER connected to 0V and 5V respectively.
* Connect the arm of a 10K pot to A0 with one of the other two connections going to power and the other to ground.
* Connect one side of an 8W speaker to D5 and the other to GROUND.
* When you get tired of the sound you can disconnect one of the speaker connections.
* Josh Miele for the Blind Arduino Project -- Sep-06-2018

//variables for measuring pulse width from servo feedback sensor
//The width of the pulse corresponds to servo position
volatile int timeHigh = 0;  //number of microS in the current cycle
volatile int prevTime = 0;  //start time in microS (from reset) of the current high cycle

//specify I/O pins
const byte servoPin = 9;  //must support PWM
const byte sensorPin = 2; //must support hardware interrupt
const byte potPin = 0;   //analog pin for reading potentiometer
const byte speakerPin = 5;  //must support PWM

//Other variables
//For measuring your feedback unit's particular min and max pulse widths
int minTime = 500;   //a selection from the mid-range to be adjusted down as the servo turns
int maxTime = 500;  //a selection from the mid-range to be adjusted up as the servo turns

//use the standard servo library
#include <Servo.h>

Servo myservo;  // create servo object 

void setup() {
  // when sensor pin goes high, call the rising function
  attachInterrupt(digitalPinToInterrupt(sensorPin), rising, RISING);
}  //end of setup fn

void loop() { 
  //use a pot to control the servo speed and direction
  //values received from the pot are between 0 and 1023
  //values sent to the servo should be between 0 and 180
  //map handles this conversion.
  myservo.write( map(analogRead(potPin), 0, 1023, 0, 180) );
  delay(5);    //for stability
  //sonify the servo feedback position
  //expect values between 20 and not much more than 1000 from the specs
  tone(speakerPin, timeHigh);  
  delay(5);  //again for stability
  //collect the min and max values for the pulse width of the feedback sensor
  //This will help us convert sensor feedback to angle for later use
  //Just let the servo spin for a while and the min and max will reach their extreme values after a few cycles
  if (minTime > timeHigh) minTime = timeHigh;
  if (maxTime < timeHigh) maxTime = timeHigh;
  printMinMax();  //see serial output function below
}   //end of main loop

void rising() {  //Interrupt service routine
  //Whenever the sensorPin goes high this routine is triggered
  attachInterrupt(digitalPinToInterrupt(sensorPin), falling, FALLING);  //reassign ISR to catch the other end of the pulse
  prevTime = micros();  //What time is it? It's safe to use micros in ISR for short durations
}  //end of rising fn

void falling() {  //interrupt service routine
  //during a pulse, this routine waits to be triggered by the drop in voltage
  attachInterrupt(digitalPinToInterrupt(sensorPin), rising, RISING);  //reset ISR to catch beginning of next pulse
  timeHigh = micros()-prevTime;  //This is how long the voltage was high in microS
}   //end of falling fn

void printMinMax() {
  //This routine prints the current min and max pulse width values on a single line 
  //Use Putty or some other terminal program to read it
  Serial.println("");  //put the newline at the beginning of  the process to stabilize braille display output
  Serial.print(' ');  //this puts a space between the numeric values

Now that you have measured the minimum and maximum pulse durations for your feedback servo, you can use the knobFBS.INO sketch below as a starting point for your own 360° servometer. Substitute your measured values for minTime and maxTime in the sketch and you should be up and running. As above, I tried to provide lots of explanatory comments in the sketch itself, so the best way to understand it is to read through it, download it, and play with it.

* This sketch shows how to control the position of a 360Deg servo with feedback.
* A pot controls the position of the servo and a speaker sonifies the servo position.
* For use with the parallax feedback 360° high speed servo
* Feedback provided via Hall Effect sensor
* Duty cycle of sensor corresponds to position with a min of 2.9% frq approx 910 Hz
* (my unit min = 24 microS, max = 1076 microS)  
* Wiring:
* Servo has a ribbon of 3 conductors and a second single wire, all with female connectors.
* With the servo shaft pointing up and the wires coming straight from the unit toward you the ribbon connections
* from left to right are GROUND, POWER, AND CONTROL. The single conductor is the feedback SENSOR.
* Connect CONTROL to D9 and the SENSOR to D2, with GROUND  and POWER connected to 0V and 5V respectively.
* Connect the arm of a 10K pot to A0 with one of the other two connections going to power and the other to ground.
* Connect one side of an 8W speaker to D5 and the other to GROUND.
* When you get tired of the sound you can disconnect one of the speaker connections.
* Josh Miele for the Blind Arduino Project -- Sep-06-2018

//variables for measuring pulse width from servo feedback sensor
volatile int timeHigh = 0;
volatile int prevTime = 0;

//specify I/O pins
const byte servoPin = 9;
const byte sensorPin = 2; 
const byte potPin = 0;
const byte speakerPin = 5;

//Other variables
//use the basicSFB sketch to collect these values for your particular unit
float minTime = 24;  //min pulse width in microS corresponding to angle of 0
float maxTime = 1076;  //max pulse width in microS corresponding to 359.99 deg

#include <Servo.h>

Servo myservo;  // create servo object 

void setup() {
  // when sensor pin goes high, call the rising function
  attachInterrupt(digitalPinToInterrupt(sensorPin), rising, RISING);
}  //end of setup fn

void loop() { 
  //use a pot to specify target angle
  int potVal = analogRead(potPin);  //expect values between 0 and 1023 from pot
  //fbServoGo expects values between 0 and 360, so use map to get there from potVal
  int targetAngle = map(potVal, 0, 1023, 0, 360); 
  fbServoGo(targetAngle);  //see function below for setting the servo position
  tone(speakerPin, time2deg(timeHigh)+360);  //sonify the servo position over 1 octave
  delay(5);  //for stability
}   //end of main loop

void rising() {  //Interrupt service routine
  attachInterrupt(digitalPinToInterrupt(sensorPin), falling, FALLING);  //reassign ISR to catch the other end of the high time
  prevTime = micros();  //What time is it? It's safe to use micros in ISR for short durations
}  //end of rising fn

void falling() {  //interrupt service routine
  attachInterrupt(digitalPinToInterrupt(sensorPin), rising, RISING);  //reset ISR to catch next voltage rise
  timeHigh = micros()-prevTime;  //This is how long the voltage was high
}   //end of falling fn

int time2deg(int time) {
  //convert pulse duration to servo angle in degrees
  return ((time - minTime) / (maxTime - minTime)) * 360.99;
}   //end of time2deg fn

void fbServoGo(int targetAngle) {
  //Make the  feedback servo point in  a particular direction
  //with the minimum of movement (don't go the long way around)
  byte speed = 2;  //larger values slow response and increase stability
  int servoAngle;  //just initializing it for later.
  int currentAngle = time2deg(timeHigh);  //gets servo position in degrees
  //calculate the 360 deg difference between current and target angles.
  int angleDifference = (targetAngle - currentAngle + 360) % 360;
  //calculate the absolute (minimum) angle difference between current and target angles
  int absAngleDifference = abs(360-angleDifference);
  //Use 360 deg angle difference to decide which way to turn (CW or CCW)
  //use absolute angle difference to adjust motor speed as you get closer to the target.
  if (angleDifference <= 180) servoAngle = 90 - (absAngleDifference/speed);
  else servoAngle = 90 + (absAngleDifference/speed);
  myservo.write(constrain(servoAngle, 0, 180)); //use constrain to keep things reasonable...

Friday, August 25, 2017

Meet The Grove Shield

The Grove Shield is a particularly useful tool for blind or sighted beginners working with Arduino. It’s awesome for quick proof-of-concept projects, introductory Arduino workshops, and for anyone with impaired dexterity or fine motor control issues. The main benefit of the Grove Shield and its family of components is that it saves you from the hassle of wiring each component to ground, power, and its respective input or output pins, simplifying project construction significantly. It also ensures solid connections with snugly fitting plugs and sockets. Only one kind of cable is used for all components, and the plugs have a clear orientation that prevents you from plugging them in backwards.

The Grove Shield from Seeed Studios piggybacks on the Arduino Uno or similar controller’s and allows easy connection to a large number of sensors, actuators, and communication options. In addition to the standard headers, the Grove Shield includes 16 sockets, all of which use the same four-conductor ribbon cables to connect Grove components to analog, digital, I2C, and UART pins on the Arduino. Each socket includes connections to ground and power, so wiring up projects becomes extremely easy. The Grove Shield also includes a Voltage Switch to switch between 5V and 3.3V power, a reset button duplicating the one on the Arduino, and the standard set of 6 male ISP pins. Oh, yeah – it also includes an LED if you’re in to that sort of thing.

A very nice Grove starter kit can be ordered from Amazon Seeedstudio Grove for Arduino - Starter Kit V3 for about $45, although it does include a relatively expensive LCD component which blind makers won’t have much use for. Other than that, the starter kit is great and includes a nice variety of components. Additional components can be ordered from Amazon or from Seeed Studios. Note that the starter kit does not include an Arduino.

When you receive your Grove Shield the pins on the back of the board are embedded in a foam pad to protect them from being bent. While getting oriented to the shield we recommend leaving the board attached to the foam or seating the shield on an Arduino. The pins on the bottom of the shield are delicate, and whenever you connect or disconnect the shield from an Arduino you should always be careful to avoid bending pins.

Orienting to the Grove Shield

With the Grove shield and its foam pad on a table in front of you, rotate the board so the Voltage switch and reset buttons are at the top-left corner, and the standard headers run vertically along the left and right edges of the board. The 6 male ISP pins (looking something like a sideways braille cell) should be toward you in the middle of the bottom edge of the shield.

The Voltage Switch faces left at the top edge of the left side of the shield. When the switch is set toward you the shield is set to use 5V power supply. When the switch is set away from you the board receives 3.3V. We have found this switch to be a little delicate, so handle it with care. If it breaks, the board is broken. Just below the switch on the left edge of the board is the reset button (and the LED). Along the top edge of the shield are the 4 analog sockets. located between the two standard headers are 12 other sockets arranged in a 3X4 grid like a telephone keypad.

Analog Sockets

The analog sockets line the top edge of the shield. From left to right they are A0, A1, A2, and A3.

I2C Sockets

The I2C sockets are the left-most column of 4 in the array of 12 sockets. Using the telephone keypad analogy, the I2C sockets correspond to 1, 4, 7, and *.

Digital and UART Sockets

The digital sockets are the top-three sockets in the right-hand column of the main group of twelve, as well as the center column of 4 sockets. The UART socket which uses digital pins 0 and 1 is the bottom right socket in the group of twelve (the pound sign in our keypad analogy).

Moving up the column from the UART socket, the sockets are D2, D3, and D4. Starting with the bottom-center socket and moving up, sockets are D5, D6, D7, and D8.

Primary and Secondary Connections

Somewhat confusingly, each analog and digital socket actually connect to two different pins – a primary and a secondary. Most components only use one of these connections, but some use both. Above we have listed only the primary connections, but each socket also connects to the next pin as a secondary connection. In other words, the secondary connection for A0 is A1, the secondary connection for A1 is A2, and so on. The digital connections follow the same pattern, with the secondary connection for D2 being D3, etc. In general, you only need to worry about the primary connections.

Orientation to Grove Sockets and Plugs

All the Grove sockets have a basic rectangular profile ( approximately 1/2X1/4 inch), and they are all oriented on the board with the longer axis running from left to right. Each socket has a notch approximately ¼ inch wide cut in the center of the two longer sides. One side has a much deeper notch than the other, and we will call the side with the deeper notch the “front” of the socket. You may have already noticed that the analog sockets are rotated at 90 degrees to the other 12 sockets. In other words, analog sockets have their fronts facing up with plugs entering from the edge of the board, while the other twelve have their fronts facing you with plugs entering from above. Each Grove component also has the exact same socket with a front and back.

The ribbon cables that connect the Grove components to the shield have an identical plug at each end. Each plug has a flat side which goes toward the back of the socket, and a side with two long parallel ridges which goes toward the front of the socket. At the very end of each plug are four holes which connect with corresponding pins inside the sockets. These holes correspond to the four wires of the ribbon cable. When the cable is plugged in to a socket with the front of the socket facing you, the wires, from left to right are primary pin, secondary pin, power, and ground, and are colored yellow, white, red, and black respectively. This is extremely useful to know if you want to integrate Grove and non-Grove components.

Friday, October 7, 2016

Bay Area Blind Arduino Monthly Meetup

The Blind Arduino Project is delighted to announce the kickoff of the Bay Area Blind Arduino Monthly Meetup -- BABAMM!! Beginning on October 8, 2016, Bay Area makers, both blind and sighted, will gather on the second Saturday of every month to construct, code, converse, and collaborate on Arduino and Arduino-ish projects. Bring your own or plan to join with others to teach and learn in a supportive and friendly environment focusing on empowering blind and visually-impaired people to make their own accessible stuff with Arduino.

If you are a blind maker working on a project, or if you are blind or sighted and have an interest in learning more about how blind and visually-impaired makers build stuff with Arduino, we invite you to bring your projects and your passion to BABAMM. We welcome beginners and masters alike, and are eager to learn and share our knowledge about all things Blind Arduino!

Meetups take place in the Innovation Lab at the LightHouse in San Francisco from 1:00 to 5:00 on the second Saturday of every month. Space is not unlimited, so it is really helpful to have an idea of how many people to expect. If you’re interested or think you might want to come, we would really appreciate it if you would sign up for the Bay Area Blind Arduino Monthly Meetup group. This makes it a lot easier for us to remind you when the meetups are happening, and to let you know about any special things that might be planned for future meetups.

Please join us for BABAMM! We look forward to crossing paths with many new and old friends as the Bay Area Blind Arduino Monthly Meetups continue to build momentum, community, and cool devices.

Tuesday, August 23, 2016

Configuring The Arduino IDE with the Java Access Bridge

Elsewhere in this blog I have discussed the relative inaccessibility of the Arduino IDE -- the free integrated development environment downloadable from Various rumors and suggestions have reached me about how it can be configured to work with the Java Access Bridge, but despite hours of effort I have never been able to do it myself.

Now, Ken Perry, blind maker, frequent advisor to the Blind Arduino Blog, and software engineer at the American Printing House for the Blind, has finally documented how to make the Arduino IDE work with a Windows screen reader. It is one of the early posts in what is bound to be an awesome blog on blind electronics.

It's great that Ken has finally told us how to get the Arduino IDE to work with the JAB, but I will probably continue to write code and upload from Notepad++ rather than use JAB with the IDE. This is mostly because I have concerns about the security of the Java Access Bridge, but I also feel like I have spent enough hours of my life fruitlessly wrestling with the JAB and don't feel like doing it any more. Maybe I'll try it some day when I'm feeling energized and long on patience.

For many people working in larger institutions such as schools or libraries, the JAB approach will also not be convenient because it requires extensive modification of environment variables and system-level files. Many system administrators will not like the configuration changes necessary to make the Java Access Bridge work with the Arduino IDE.

But the good news is that you can do it if you want, and Ken Perry is the hero of the day for finally telling us how. Thanks, Ken!

Sunday, July 3, 2016

How to Compile and Upload Arduino Sketches with Notepad++ -- Simple, Convenient, Accessible


Blind people living in the real world are constantly working out hacks and alternatives to “normal” work flows that are inaccessible in whole or in part. Arduino software development is no exception, and many of the posts in this blog have been devoted to working around the sketchy (sic) accessibility of the Arduino IDE itself. For example, in an earlier post I describe (perhaps in more detail than strictly necessary) how to set up and configure the Arduino IDE from the Windows command line. In another post, Chancey Fleet provided excellent step-by-step instructions for installing and using Visual Micro to compile and upload Arduino sketches directly from Visual Studio. The current post offers the most convenient Arduino development tool chain I have yet found –editing, compiling, and uploading sketches directly from Notepad++.

Notepad++ (hereafter Npp) is my text editor of choice for almost any simple editing work including coding Arduino. It is used by many developers who don’t want the overhead of a giant IDE just to edit a bit of code. It’s free, open source, simple to install, and largely accessible with whatever screen reader you happen to prefer. It has features such as code completion, line numbers, support for different character encodings, and many others, yet it remains extremely compact and easy to use. Perhaps my favorite feature is that it doesn’t care what kind of line endings you use, CR/LF or just LF – it just does the right thing. This is really nice when using cross-platform source files such as Arduino sketches. If you don’t know what I’m talking about, just take my word for it: Notepad++ is extremely convenient and easy to use as a code editor.

Compiling and uploading Arduino sketches from the command line is great for those of us who are really old school and feel at home typing incredibly long text-based commands. For others though, those long commands are just opportunities for typos and inexplicable DOS error messages. Similarly, for hardcore hackers who have millions of lines of code under their belts, Visual Studio might be the best Arduino development environment you could want. Speaking for myself, however, Visual Studio makes me feel like the Sorcerer’s Apprentice with too many options, settings, and terms that I don’t quite understand, and an incredibly complex interface as reins to ultimate unwanted and undeserved power. In fact, I recently hit an accidental keystroke in VS and the entire thing stopped working. Of course, if I knew what I was doing in there then I’d be able to fix it, but I don’t have that kind of time and I don’t want that kind of power. I just want to write a little Arduino code!

In contrast, Npp is just my speed. It lets me do what I need to do without giving me a dangerous amount of rope. At the same time it has hidden depths which can be quite handy. You could say that I am definitely down with Npp…

But Npp is just an editor. It is not a compiler or an integrated development environment, so it can’t compile or upload your Arduino sketches. On the other hand, it can be configured to interact with other programs on your computer (such as the Arduino IDE) which can. This article explains how to set things up to let you edit an Arduino sketch with Notepad++, and then compile and upload that sketch with a single command without ever leaving Npp. The resulting Arduino compiler output and error messages are even displayed in a Notepad++ window for easy review.

The following sections offer two scripts that can be executed from inside Notepad++. The first allows you to set the Arduino COM port – convenient since this may change frequently with different boards and USB ports. The other compiles and uploads the Arduino sketch that is currently open in Notepad++.

Setting It Up

Step 1: Download the latest Arduino IDE and install it. If necessary, check out my instructions on installing and configuring the Arduino IDE from the command line.

Step 2: Download the latest version of Notepad++ (Npp) and install it. This process is easy and you should have no accessibility-related problems. Take note of the directory where you are installing it because you’ll need to find it later to install the Npp plugin.

Step 3: Download NppExec (a plugin for NPP) and install it. The installation process consists of unzipping the NppExec archive and copying the appropriate DLL into the plugins directory of the Npp installation directory. You are probably using the Unicode version of Npp, so copy the NppExec.dll file from the DLL_Unicode folder of the archive to the plugins directory of Npp and restart Npp to complete the installation. For example, my Npp plugins directory is "C:\Program Files (x86)\Notepad++\plugins"

Step 4: Set up the following Arduino-related NppExec scripts in Npp. You can either copy and paste them from below or download the NppExec scripts in this convenient archive.

The easiest way to install these scripts is to copy and paste them individually into the Execute dialog in NPP. This dialog can be opened with a keyboard shortcut (F6 inside Npp), or through Npp’s menus (Plugins > NppExec > Execute…).

The Execute dialog has a multi-line edit field for pasting or typing the script itself, a Save button for naming and saving scripts, a combo box for recalling previously saved scripts, and an OK button to execute the selected script. All the controls in this dialog are accessible with NVDA or JAWS.

The Arduino COM Port Script

This script makes it easy to quickly update the Arduino IDE’s COM port. This is the port used to upload the sketch to the board. When you invoke this script from inside Npp, a dialog pops up asking for the Arduino’s COM port. Appropriate responses are like the parameters you would use on the command line when setting the com port (e.g., com3, com4, etc.).

To set up the port selection script, open the Execute dialog by pressing F6 or by selecting Execute from the NppExec sub-menu of the Plugins menu. Paste the following script into the edit field and press the Save button. In the Save dialog, type something like “ Arduino Port,” and press return. The port selection script will then be saved for future use using that name.

Before saving, be sure to edit this script to reflect the actual location of your Arduino installation. I have mine in the root directory for convenience – “c:\arduino”.

//This NppExec script sets the com port of your Arduino IDE
//By Josh Miele -- June 27, 2016
//The Blind Arduino Project
set arduino_path = c:\arduino\arduino_debug //the path to your Arduino IDE.
//put up a dialog box requesting com port name
inputbox "Please enter the COM port of your Arduino (e.g., com5):" //gets com port value
//use result to set port value doesn't verify
cmd /c "$(arduino_path)" --port $(input)
npp_console 0 //hide the console

Note: This script does no checking to make sure that you have entered a valid COM port or string. If you make an error typing or enter an invalid com port, you will get no error message or other feedback to indicate your mistake. Take care!

The Compile and Upload Script

This script saves the currently open Arduino sketch, then calls the Arduino IDE with the command line parameters to compile and upload it. It then passes any output, including errors to a new Npp file so that you can easily see what happened.

To set up the compile/upload script, open the Execute dialog by pressing F6 or by selecting Execute from the NppExec sub-menu of the Plugins menu. Paste the following script into the edit field and press the Save button. In the Save dialog, type something like “ Arduino Upload,” and press return. The compile/upload script will then be saved for future use using that name.

Before saving, be sure to edit this script to reflect the actual location of your Arduino installation. I have mine in the root directory for convenience – “c:\arduino”. Also make sure that the Arduino_output.txt file is being created in a directory where you have write privileges and where it won’t cause any problems. I put mine in “c:\temp”. The first time the script runs it will ask for permission to create the output file. After that it will quietly overwrite the file with each new execution of the script.

//This NppExec script compiles and uploads the current Arduino sketch in Npp
//By Josh Miele -- June 27, 2016
//The Blind Arduino Project
//Set where the output goes. Make sure you have write privileges
set arduino_output = "c:\temp\arduino messages.txt" 
//Set location of Arduino executable. 
set arduino_path = c:\arduino\arduino_debug 
npp_save //save current file before uploading
//Compile and upload sketch in current Npp window and send stdOut and stdErr to arduino_output
cmd /c "$(arduino_path)" --upload $(full_current_path)>$(arduino_output) 2>&1
npp_console 0 //hide the console
//show the results of the compile/upload in Npp window
npp_open $(arduino_output)  

Note: This script does not account for the fact that the Arduino IDE will move your sketch if it is not already inside a folder of the same name as the stem of the script. For example, if you are editing a file called MySketch.ino located in c:\temp, executing the above NppExec compile/upload script will compile and upload the sketch, but the Arduino Ide will create a directory called c:\temp\MySketch and move the file MySketch.ino inside it. Notepad++will be left with an open file that has been moved out from under it. This has the potential to lead to unexpected errors and problems, so I recommend making sure your sketches are contained within folders of the appropriate name as modeled in the Arduino examples directory.

Using the Scripts

To invoke these scripts from inside Npp, press F6 to bring up the Execute dialog, then use your arrow keys to select the desired script from the combo box and press return. Pressing ctrl-F6 will execute the last script without invoking the execute dialog.

Note: There is a way to map individual NppExec scripts to their own keyboard shortcuts using the Shortcut Mapper under the Settings menu. Unfortunately, this dialog is not accessible using NVDA or JAWS. I believe there is a way to modify the shortcut mappings by editing one of Npp’s XML files, but I don’t yet know how to do this. In the interest of getting this info into your hands, I decided to post this anyway and update the post when I find an accessible way to map the shortcuts. If you have info about how to do this, please leave it in the comments.

References and Disclaimers

I’m no expert in any of the several areas necessary for creating elegant and effective NppExec scripts. I encourage feedback in the comments and invite suggestions for improvement and additional helpful scripts.

In developing these scripts I found the following resources extremely informative: