LinuxFest Northwest Presentation on Open Source car entertainment

Here's my presentation at Linux Fest Northwest. The focus of this presentation moved away slightly from my previous presentations on user experience to doing automation and other control of the vehicle, getting information and the tools required to do such. I talk about proximity, obdgpslogger and nobdy and what you can do with them to automate things in your car, gather statistics, and have fun!

Hope you enjoy it!


- Slides

Audio:



There is a part of the audio that doesn't have matching slides (the part where I give the live demo). To describe it, I basically had bluemonkey running on a desktop at home (portland) with nobdy and obdsim from the obdgpslogger project. I was communicating with bluemonkey over IRC and was able to get the power state, secondary battery voltage, the vehicle state, statistics from the car (speed, RPMS, MPG) and determine where my car was (bluemonkey replied to my query with a link to google maps showing it's current location).

Introducing: WifiProximity

Ever wanted to synchronize something based on the availability of a wifi access point? WifiProximity is the solution.

WifiProximity is a wifi access point proximity based generic action processor. In other words, it does "stuff" when it sees specified access points. The actions can be any number of system commands like rsync, touch, finger, etc. WifiProximity is particularly useful for netbooks, mobile internet devices, etc where you may be moving back and forth between different access points (lets say the home and office) daily.

The idea comes from blueproximity, an app for locking/unlocking your machine based on the proximity of a bluetooth device.

WifiProximity is written using Qt and NetworkManager's DBus API to detect and connect to access points. It's actions are scriptable. Here is an example event script:

WifiProximity is still a young project, but hopefully it will become useful in mobile computing. It is currently a part of the OpenICE project. You can find the code for WifiProximity here:

https://linuxice.svn.sourceforge.net/svnroot/linuxice/packages/wifiproximity

The mobilest of environments - the car

I was recently involved in a conversation on a forum thread discussing why current netbook and mobile editions would not be optimal for In Vehicle Infotainment (IVI):

It should also be noted, IVI interfaces differ from the mobile interfaces found in phones and MIDs. Those interfaces are still designed with "captive attention" in mind. IVI interfaces cannot demand captive attention by nature. They are close, but the differences do matter. Thus anything that your run on your mobile device in the car will have to be designed for that purpose. You can still get away with running mobile apps in IVI environments, but it's probably not going to be optimal.

"Captive attention" means that if you are using your phone, your phone is naturally the only thing your attention is on. You are typing a number, text message, navigating your music or other menus, and doing so while looking at the screen.

In the car, you are briefly looking at the screen, touching something, and then re-focusing your eyes where they should be: ON THE ROAD. This is why a mobile environment for the car has to be designed with that in mind. The text must be large enough for you to read at a glance. Navigation must be simpler. Buttons must be large. These are some of the concepts in the OpenICE standard. For more information about what makes a good mobile enviroment for cars, I'll refer you there.

Most mobile systems are not created with this in mind. These systems cannot be run optimally in the car and that is one of the big reasons why many state laws require only "hands-free" phone use in the car (please, please do not text and drive...).

Multi-Process Mobile Desktop Concept

With the introduction of the Atom 330 processor, dual-core computing was realized in the mobile world. While still keeping the overall power consumption down, these chips add real power and additional capabilities to the system.

In order to take advantage of this new power, software must be designed with multi-threading in mind. Graphical applications that use a GUI toolkit, often perform background work on one thread, and run the Graphics on another thread. Most of the time, multi-threading can be a headache as developers run into race-conditions, deadlocks, and other problems related to multi-threading. Many times multi-threaded apps are discouraged and avoided by some developers.

In addition, when a thread in an application crashes, it usually brings the entire app down with it. For plugin-centric applications like firefox, this can be a huge annoyance to the end-user when a little ad-blocker plugin brings down your entire browsing session.

So how does a developer take advantage of the multiple core processing power and avoid the issues related to multi-threaded development? The answer may be a multi-process design.

Google Chrome wasn't the first to implement a multi-process design, nor will it be the last. Each tab in the Chrome browser is a separate process. Google did this to increase stability and security. And it works. When a tab crashes in Chrome, the tab dies... Chrome keeps on chugging.

Should the same concept apply to mobile computing? I don't see why it shouldn't. The hardware is becoming cheaper, and multi-core chips are here to stay.

A Multi-Threaded Mobile Desktop

To start out with a mobile desktop, one must first understand the mobile environment. Mobile devices usually hand held, or dash-mounted (in the car), with anywhere from a 4"-8" screen. Because the screen is smaller, fonts and other graphical elements need to scale so they are readable by the user. Applications on these usually run full screen.

The desktop in a mobile environment also differs slightly from a desktop on a personal computer. A PC desktop is usually a place where you can put files, and launch applications. It is usually run by the file-manager like nautilus in GNOME and explorer in windows. A mobile desktop is also a place where you can launch applications. But it probably isn't a good place to dump files on because of the lack of screen space. In reality, the mobile desktop is an application that launches other applications. For our multi-process desktop, we will take this idea one step further.

Building a mobile desktop

Writing apps takes time. Writing plugins using a very specific set of tools and interfaces is often much faster. The desktop in our example will act as a server: it will facilitate communication between plugins. Launch them at the users request, and close them when they are no longer needed. Below is a c++ example of a simple plugin interface:

class PluginInterface
{
public:

PluginInterface();
virtual ~PluginInterface();

virtual void OnCommandRecieved(CommandClass command) = 0;
virtual void init(DesktopServer ServerInstance) = 0;
virtual void run() = 0;
};

Why is communication needed? In a mobile environment, resources are scarce. The audio device is an excellent example of a scarce resource. If you are using your mobile GPS to guide you through traffic while listening to music, you don't want your directions to mix with the music. Not only will you not understand where to go, your music enjoyment will be interrupted by a mix of disharmony and chaos. The GPS application/plugin, needs to ask for control of the audio device. It can send a message to the server to "fade" the music volume down to 0, pause, and then fade back in the music when the GPS is done giving it's direction. That sounds much better than Ozzy Ozborne being interrupted by "Turn Left in 30 meters".

By using DBus, a common *nix IPC messaging system, plugins can communicate with each other, and with the server.

Plugins, however, are usually in-process code. How do they fit in a multi-process environment? Simple, they don't. They need a "harness" in order to run as a separate process. In the case of our mobile desktop, a lightweight "core" will be used. The main function of this "core" is to setup communication with the server, load the plugin, and make sure the plugin gets whatever the server sends it. Below is an simple example of our core application:

int main(int argc, char** argv)
{
if(argc < 2)
printf("usage: core [pluginname]");

string pluginname = string(argv[1]);

PluginInterface *plugin = LoadPlugin(pluginname);

DesktopServer *Server = new Server();

Server->RegisterPlugin(pluginname);

plugin->init(Server);
plugin->run();

}

In our desktop, we will have a dock plugin, an app/plugin launcher plugin, an audio device manager plugin and a notification plugin. When any additional plugins or apps are loaded, an icon will appear on the dock so users can switch between that plugin and others. When a plugin wants to use the audio device, it will ask the audio device plugin for control. Likewise when a plugin wants to notify the user of something, like an incomming call, it can send a message to the notification plugin.

Here is an example of a command being sent to the audio device plugin:

void Speak()
{
string volumelevel = Server.Evaluate("AudioDevice:GetVolumeLevel");

Server.Send("AudioDevice:Fade '0'");
Server.Send("AudioDevice:Pause");

SpeakSomething("Turn Left in 100 Meters");

Server.Send("AudioDevice:Resume '0'");
Server.Send("AudioDevice:Fade '"+volumelevel+"'");


}

The server then parses the message, grabs the plugin name "AudioDevice" and forwards the message to the Audio Device plugin:

void OnServerMessageReceived(string message)
{
string plugin_name = GetPluginName(message);
Plugin plugin = GetPlugin(plugin_name);

plugin.Forward(message);

}

Below is a visual example of the server core+plugin concept:



When it's all run together, the result is a tightly integrated stable and secure desktop system like the visual below:





Concept Drawbacks

It would be dishonest to present a concept without pointing out the drawbacks, obvious or not. The first drawback is that applications that are used in the system that aren't plugins or do not have plugin-wrappers have a huge disadvantage. These applications won't be able to integrate at the level the plugins will. They likely won't know about the notification plugin, or the audio device plugin. Second, this system forces the developer to learn yet another plugin system, etc.


Conclusion

A multi-process desktop has many advantages. Stability: plugins can't crash the desktop or any other plugin; Security: precautions made by the OS help partition what the plugin can do; Integration: plugins can talk to each other to provide the ultimate user experience; Rapid Development: plugins are faster to develop than entire apps.

Finally, OpenICE, the Open In-Car-Entertainment group plans on implementing the multi-process desktop for use in in-vehicle infotainment systems in the 2.0 release in 2010. Stay tuned...

OpenICE - Computer Enhanced Cars

No matter how you look at it, technology in cars has come a long way since the cassette tape. When you buy a Car today, you most likely were offered by the dealer an upgraded "technology package". This usually includes upgraded speakers, GPS navigation, Bluetooth and the ability to play music from your iPod.

With all the technology available today, do these enhancements really help enhance our experience. What is the future of technology in cars? I hope to point to a opensource software project that looks to answer the question.

There are many closed-source software solutions looking to become the center of In Vehicle Infotainment (IVI). Streetdeck an advanced car interface for the Windows XP operating system just released their 2.0 version last month. Centrafuse, another software application for Windows was seen on a number of devices at CES this year. Many of us have seen the Microsoft/Ford Sync commercials on TV. So where is the Open Source alternative? Enter OpenICE.

OpenICE (Open In Car Entertainment/Infotainment) is a open source project that aims to create a standard for IVI systems. OpenICE also develops software to meet these standards.

OpenICE categorizes IVI into 4 sections:

* Entertainment
* Information
* Connectivity
* Mobility

Entertainment:
OpenICE maintains a "Plays Everything" policy by utilizing open source technology. Unlink many closed-source systems, the OpenICE media playback software, nGhost Media Desktop, can play back a number of non-patent encumbered media codecs like Ogg-Vorbis/Theora and FLAC. NGhost Media desktop also supports plugins for mplayer and phonon, which allows backends such as gstreamer among others to be used.

The Future:
The future of entertainment is in smart devices. Smart phones, mobile internet devices, and Ultra Mobile PCs are examples of such devices. In the future, OpenICE hopes to integrate with these devices so that the moment you walk into the car, the car recognizes these devices, and the car's interface displays and controls information from the devices. Drivers will be able to play mp3's from their phone on their car stereo system via bluetooth or wifi. They'll be able to answer phone calls by voice activation allowing the driver to focus on driving.



Information
OpenICE has developed plugins to the nGhost media desktop to give real-time car statistics to the driver. In the current version of the plugin, the driver can view Engine RPM, Speed, Engine Coolant Temperature, and Engine Load.

OpenICE has also developed plugins to display weather and traffic information to drivers.

The Future:
OpenICE hopes to extend information into read even more data from the vehicle, including error codes. OpenICE hopes to integrate traffic information into the navigation and automatically reroute you to avoid traffic.

OpenICE also hopes to integrate automation into the vehicle. By selecting a user preference, the car may automatically roll down windows and shut off the air conditioning to save energy when traveling at certain speeds. The volume on your music may automatically adjust to compensate for wind noise when traveling at high speeds. The vehicle may warn you that you are driving to aggressively and recommend different ways to save gas by adjusting your driving behavior.

Connectivity
OpenICE hopes to achieve connectivity with mobile devices. Handsfree communication is essential in IVI environments. OpenICE uses the nohands project to provide handsfree communication with cellular phones with bluetooth.



Streaming media over bluetooth, upnp via wifi, are also ways in which OpenICE hopes to connect the driver to his mobile devices.


Mobility
The Vehicle is a very different environment then most. The driver must keep his attention on the road where it belongs. Any technology added to the car must aid the driver and not distract him.

OpenICE implements guidelines for graphical interface so the driver will be able to use the system, and still keep his attention on the road.

Currently mobile graphical interfaces are designed for use in handheld devices. Because of the requirements of the Vehicle's environment, where the drivers attention is only on the interface for brief moments, no other mobile graphical interface existed that really tailored to life on the road. OpenICE developed it's own customized desktop (nghost) and dock (icepanel) to satisfy these specific requirements.



Video of nGhost,icepanel, and Compiz-Fusion's cube rotate:


Conclusion
OpenICE is a young project with big goals. It recently (yesterday) released it's development roadmap where hopefully, many of these future ideas and concepts will become a reality. As with any young open source project, the more developers that join and contribute, the better the experience for everyone will be who uses the software.

Hopefully, we will soon see some aftermarket products based on the OpenICE platform. Eventually, even auto makers will include Open Standards in their "technology" offerings.

Overclocking your CPU in Linux

While tools for overclocking your CPU seem to be more available and spoken of on the windows side, here are a few useful tools that I've found useful while attempting to overclock my cpu in Linux.

SysInfo
This app is very useful for seeing just what Frequency your cpu is running at. It's no "CPU-Z" but it does a fairly good job.

ComputerTemp
As your cpu temperature rises as a result of overclocking, it's necessary to monitor the cpu's temperature. Computer Temperature Monitor is gnome panel applet that monitors temperatures as reported by several sensor on you PC. You can even use it to create alarms if the temp goes too high.



Super PI
Yes, the tool often used to benchmark systems running windows runs perfectly fine in WINE. Super PI is a good benchmark to see what kind of improvements you are achieving by overclocking. It will only utilize one core. Using this will help provide a common ground for comparing windows based machines with yours.

time
The good old *nix command "time"... Time outputs the length of time it takes to run a particular application. For my load testing, which I do to make sure the system is stable enough at the speed I'm overclocking it at, I compile my own project and time it. Timing it allows you to use the end result as a benchmark. You can use time on any CPU intensive application, such as transcoding video or audio, kernel compiling, etc.

Note, if you do end up compiling something, be sure to use the "jobs" argument when typing "make". The general rule is that you want the number of jobs to exceed the number of processing cores plus one. In my case, with an AMD Athlon X2 5400+, I used "time make -j3", -j being the number of concurrent jobs to run.



Happy overclocking!!

skype + dbus + mono

Once again, in effort to create yet another plugin for nGhost I've tread where no man has trodden before, skype + dbus + mono. Once again, because ndesk.dbus has excellent support for dbus, c# was chosen.

The skype API is a set of string commands that you send to skype to do various functions. You can do a lot through the API including sending SMS messages.

At the time of writing this, the dbus documentation for skype was definitely lacking. I was particularly confused on whether the method "Notify" was a dbus-signal, or a service that I had to implement myself. It ended up being the latter. Here are the interfaces:

[Interface("com.Skype.API")]
public interface SkypeSend: Introspectable
{
string Invoke(string message);
}

[Interface("com.Skype.API.Client")]
public interface SkypeResponse: Introspectable
{
void Notify(string message);
}

The "Invoke" method is for Client to Skype communication. Answers to queries are returned back from "Invoke"

Notify, as discussed earlier, is for Skype to client messages. Messages such as Call ... Ringing, or Chat messages are received here. This is a dbus service you have to register yourself using the same dbus connection as you created for "Invoke".

Here is the classes that implement the interfaces:

public class nSkypeDbus
{
public nSkypeDbus()
{
init();
}

public void init()
{
dbus = Bus.Session;
skype = dbus.GetObject("com.Skype.API", new ObjectPath("/com/Skype"));
send("NAME nskype");
send("PROTOCOL 7");
}

public string send(string message)
{
return skype.Invoke(message);
}

private SkypeSend skype;
public Connection dbus;
}

And the implementation of the Client interface:

public class SkypeClient: SkypeResponse
{
private Connection dbus;

public SkypeClient(Connection d)
{
dbus = d;

dbus.Register("com.Skype.API",new ObjectPath("/com/Skype/Client"), this);
NotifyEvent += new NotifyEventHandler(InterpretNotify);
}

public void Notify(string message)
{
System.Console.WriteLine(message);
if(NotifyEvent!=null)
NotifyEvent(message);
}

public void loop()
{
dbus.Iterate();
}


public string Introspect()
{
return "\n \n \n \n \n\n";
}

public void InterpretNotify(string message)
{
if(message.Contains("STATUS RINGING"))
///sample message: CALL 1412 STATUS RINGING
{
if(IncomingCall!=null)
{
string [] p = message.Split(' ');
string caller = p[1];
IncomingCall(this, caller);
}
}
}
public event NotifyEventHandler NotifyEvent;
public event IncomingCallHandler IncomingCall;
}

In the Client implementation, I am trying to catch incoming call notifications.

There's plenty of stuff you can do with skype. Here's the link to the API: http://share.skype.com/sites/devzone/2006/01/api_reference_for_skype_20_bet.html

Hope to see more skype goodies for Linux....