039 - Adding real-time processes to on/when programming?

This note will try to discuss if it is possible (or reasonable) to add safe real-time processes to systems like the nRFGo Studio by Nordic Semiconductor [1], or to the Echelon Lon-controller, programmed in Neuron C [2]?

(After having almost finished this note, it has swollen to show how a channel based system could be written, more generally.)

Neuron C is used in Neuron Chips and Smart Transceivers, where Network variables are fundamental to Neuron C and LONWORKS applications. The nRFgo SDK is a Software Development Kit for Nordic nRF24L Series 2.4GHz RF System-on-Chips (SoCs).

Both have event-driven (on/when) architecture. Add "safe processes" to already well tuned software and hardware architectures, what do I mean?

Should you follow this note as it develops, please warn me if I say something wrong. To say it shortly, I would know my mantra (write a CSP channel based process scheduler in C) quite well. Many blog notes, publications and invited lectures revolve around that theme. But nRFGo and Neuron C I have never had any experience with. I simply rememeber long ago (about 1995) when some collegues did use the Neuron C. I see it's still alive. And the nRFGo seems to have been written just up the hill from where I live, so I got curious.

I don't know if an nRFGo or Neuron C programmer would ever want or need a process term. Since it would be the first I would personally look for, I would even before that have a look at what's already there. I have programmed so much microcontrollers that I would know what an interrupt is there for. An interrupt function, is a sort of "process", since there is microcode (or whatever) to handle how it is able to set other functions aside, and later on return to the interrupted function the morning after. Only, the interrupted code slept without knowing it.

And I remember around the year 2000, when I was on a committee called RTJWG [3] (now defunct), how so much of the discussions was about how to connect asynchronous I/O to the Java language. I had programmed the transputer for almost ten years at the time, and it and the occam language didn't even have a separate interrupt term. All was processes. So, I really didn't understand why they were discussing this so much then. I also remember a programmer I worked with back in the early eighties who got that part almost all wrong, and a collegue had to spend a year dismantling and collecting the assembly code back again. The part of the product that had a run time scheduler (a separate board and processor with code also written in assembly) worked from day one (in the asyncronous I/O and process term context). We got it all working according to specs before some Swedish nuclear power stations had to rely on the system.

Here comes on or when programming, to deliver a framework to connect interrupts (or all internal events) to non-interrupt code. Or to decouple the two. It might not be a good idea to do an FFT in the interrupt, it may be better to just pass over the data and let some other code do it later on. The sooner the better, probably - but not in the interrupt. The interrupt would have some hard real-time schedule to it, like get it over in 20 us, because there are other interrupts, you know.

In nRFGo there already is an on (like on_pipe_updated or on_transaction_finished) and a subscriber term, and there is a dispatcher for handling of events and communication between different modules. I will later try to figure out what a module is, and what a transaction is. The nRFGo uses the Keil compiler, probably wrapped up by some tools that they developed in Qt. I think their Help tells me this. So, they seem to have a C compiler there. Even if their tool also builds code by the set-up config, the user has C available. After ARM bought Keil, the ARM suite was moved over to the Keil IDE. But Nordic lists the Keil C51 C compiler, so the core must be C51 based.

In Neuron C there is a when, and events are used in when-clauses to enable the execution of a when-task, using a special syntax. A when statement can contain more than one when clause.

I think these two mechanism basically do the same: deliver a framework to connect asynchronous I/O to the rest of the code.

Scheduler and dispatcher

nRFGo contains a dispatcher, and Neuron C a scheduler. The Wikipedia article says that a scheduler typically decides which process is running, while the dispatcher makes it running [4]. This note will not be precise about this, as it is rather difficult to distinguish in a microcontroller. nFRGo and Neuron C will do about the same, with different wording - even if Neuron C's scheduler also have preemption mode.

Towards a process model

The modules or tasks mentioned are meant to be independent subscribers to defined services. In a way they are a means to order the asynchronous I/O.

However they don't seem to be processes as I would think of them. I will try to explain.

They are closely connected to the hardware in the processors. None of the code would be portable. I remember when we wanted to port some embedded code that implemented a software UART from an AVR to and ARM, I wrote an interpreter below the hardcoded use of registers in the AVR. So, on the ARM, the interpreter picked up bytes at named memory locations and wrote back. And the oversampling represented in the UART and its use of timers, input capture and output compare registers etc. was handled with a single timer interrupt on the ARM. We didn't change a line of code on the AVR, and the ARM addition was a layer below. This layer handled two usch UARTS, one more than was available in the AVR.

The more some code handles particular hw in the processor, the less it is portable. The sw described in this note connects to that hw alone. Nothing else, as far as I can see.

There are no independent processes or tasks that may talk with each other, and have a life of their own. And since these are not defined, there is no communication or synchronization model either.

One of the Lon chips "also supply one binary semaphore to support synchronized access to shared resources between the application and the ISR processor". This is interesting, and probably very useful. But it does not add the process model I am looking for.

The process model

The process model I would suggest to add is "standard" CSP type processes with safe buffered channels. I have used much of my professional life with this model, starting with the occam language. I have also published a lot about this. Blog note 034 has something about buffered channels. Add this paper (it's also [5]), and you'd have a good starting point. I will not re-refer to these papers. There will also be rationale for this type of programming in those references.

It starts to dawn on me how difficult this note could become, or has become. I had not thought that I should include the code to anything like BufChanSched (buffered channel based process scheduler), you must do it yourself. But maybe my writings could be of some inspiration.

In my experience this model is useful both for typical data-flow dominated systems (like regulating systems and more process-oriented designs) as well as control dominated systems (like drivers or protocols). And then, most systems I have seen are a mix of both styles.

The process model by how to make it

Still, here are some points (which basically says a lot about the process model):

1. A process is a C function. Start all process names with P_.
2. Make a list of pointers to those functions.
3. No static variables in the functions.
4. No reference to external variables except the channels it uses.
5. A channel sends data and is basically blocking. The second on the channel (sender or receiver) continues to run, while the first becomes ready.
6. A channel sends signals (with no data) asynchronously (never blocks).
7. A buffered channel never blocks, but always returns an ok/full bool. When it's full the scheduler uses a channel ready signal channel to tell that the channel again may be sent to
8. A channel input may define three types of timers: ALTTIMER, EGGTIMER and REPTIMER.
9. If you need to pass parameters to a function, there is no way to do it in C. Since it's going to be called by its pointer when it's scheduled, adding parameters may be done with some kind of table. But this is most often not needed for a small scheduler in C. Since there is no language support, it's possible to get around it.
10. ...
11. You need a standard scheduler in C, that keeps a list of the entry points to all processes. The entry points are assigned once only.
12. A process has initialization code, and a "while (true)" loop.
13. It has a "proctor table" described in the reference above, that by a simple, hidden goto gets the process to any rescheduling point. See here.
14. A process has only one reason to be scheduled.
15. A process may send on a channel, but there is no output guard.
16. A process may receive on a single channel.
17. A process may wait on a list of channels in an ALT. Each component may have a boolean guard to that it's easy to switch this on and off in the process. If false that component is skipped. When one component of the ALT has become activated (is that the word?), then the ALT is torn down, so that all the other entries's "first" state is nulled.
18. You could start "individs" of each process, but then you would have to parameterise the channels.
19. Each process has a struct of local variables which is allocated on the heap by a malloc in the initialisation part. Call this the "context". This is never freed, so it's easy to know if you have overflowed.
20. ...
21. A simple return from the process code gets you down to the scheduler. Therefore, no blocking calls are allowed from any subroutine level.
22. When the scheduler finds no cause to schedule a channel or there is no channel waited for, there is a system fault. You probably have a deadlock.
23. To avoid deadlock, use a deadlock free pattern, like knock-come from blog note 009.
24. No external functions may call any function in a process file (except the scheduler).
25. The process context is local to a process, and not seen elsewhere. Only parts of it are exposed to the channel (place and sizeof).
26. Two processes may call a common function, since this scheduler is not preemptive. But do it seldom, and in no case let that function have any side effects. This means that it cannot alter common state (like registers or static etc.). The exception is all channel handling, ie. all scheduler functions. Of course it has system side effects, but not application side effects.
27. Channel size is dynamic, but the sender and receiver must agree. So a common protocol definition header file must be made. A protocol is a collection of structs with tags and data.
28. With this system communication and synchronization is the same. If you use buffering, it may become the same.
29. Channel names are handled by en enum, values from 0 to n. Channels may have to be initialized. I do it in a separate function, that would init all.
30. ...
31. Coding the ALT is the most complex, because you may need a bitset (any primitive word size or array), used by each ALT to represent the ALT set. An ALT is "built" or "tore down". You need it so that the first that arrives on the channel, only refers to that bitset to tear down, with no knowledge of the other components.
32. If you need "fair ALT", then you could bundle the ALT set in an array, and start the ALT with an index which is one more than the one just "taken". The index needs to wrap around in modulo n
33. If you need processes to have priority, think twice (priority inversion etc.).
34. If you have thought twice, make one ready queue for each priority.
35. But maybe you would rather think of channels as the means to supply priority. Should they instead be prioritized? But do think twice on this too.
36. If you do need priority, play on team with the interrupt functions. The chip designers would have thought out a scheme for you.
37. Use macros to make it code better readable and hide parameters that don't need to be seen. But be aware of limitations, like Neuron C does not support #if or #elif. The synchronization points described in [5] uses labels thas absolutely should be invisible.
38. ...

These points are well documented in the literature. But a list like the above may still be ok.

So what about nRFGo and Neuron C?

The challenge is to add a small and safe run-time system with light-weight processes to these already small systems. A hypothetical BufChanSched would probably just add a few kB of code, and around 10 bytes per blocking (zero-buffered) channel. Process overhead is also small in code and RAM. And scheduling and rescheduling is light weight, smaller than interrupt context switching.

Maybe it would be nice to have processes available also for the application code in a radio chip like nRF24LE1 or the networking Neuron chip?

Perhaps someone could make an open source project for a pluggable channel based scheduler? With it, the (right) process model comes creeping all by itself..

..provided you also have a glimpse at where it came from: occam and occam-π. And assure your boss (and/or better half) that it's not going to be a waste of knowledge, as these days you'll even make a sneek peek into some of Go.

References

[1] - http://www.nordicsemi.com/eng/Products/2.4GHz-RF/nRFgo-SDK. Is there a public manual like the Neuron C manual? So I have guessed my way.. However, there is a little used public talk forum, see http://www.nrftalk.net

[2] - http://www.echelon.com/support/documentation/manuals/devtools/default.htm (Search for "Neuron C")

[3] - Real Time Java Working group, J Consortium (Java Consortium) - (NIST), backed by HP, Microsoft, Newmonics. All remains of this seem lost. Of course, the Sun based initiative JSR-1 won (http://en.wikipedia.org/wiki/Real_time_Java)

[4] - http://en.wikipedia.org/wiki/Scheduling_(computing) dispatcher / scheduler

[5] - "New ALT for Application Timers and Synchronisation Point Scheduling" by Vannebo and Teig. See http://www.teigfam.net/oyvind/pub/pub_details.html#NewALT

.

038 - Concurrent programming paradigms all created equal to IEC 61508 functional safety standard?

April 2013 moved to http://www.teigfam.net/oyvind/home/technology/065-iec-61508-and-programming-paradigms/
.

036 - WYSIWYG semantics

This blog note has been moved to http://www.teigfam.net/oyvind/home/technology/079-wysiwyg-semantics/ on 17Jan2014

035 - Channels and rendezvous vs. safety-critical systems

New 19Dec2011, updated 3Jan2017

Moved to http://www.teigfam.net/oyvind/home/technology/035-channels-and-rendezvous-vs-safety-critical-systems on 11April2017

034 - Output guard vs. "channel ready" channel

Last edit: 16. August 2012 (updated with https into my pages 31Jan2026)

On the airplane home, the other day, I came upon an idea that I thought would make life easier. When I told a guy at work, he said, "oh, it's already in Linux".

I assume that you're somewhat familiar with "channels". In this respect I was raised with the occam language. (You would find Wikipedia-articles for most of the names I'll throw here. So I won't get too deep into some details.)

Here, a channel is based on CSP (later), and it is one-directional, non-buffered and blocking. It's even one-to-one. It connects tight processes to each other. It also allows threads to be equally tight. So, the first process that comes to a channel (be it sender or receiver), must be descheduled and not ever run again before the second process arrives on the channel. At that time there's a memcpy of the correct data from the sender's private context, to the receiver's matching field. It is possible to make it type safe. The message is passed across, while both sender and receiver are still. In Ada it's called a rendezvous. In Go it has no name. When the memcpy is done, usually the second on the channel just runs on, while the first is put in the ready-to-be-scheduled queue.

There is a mechanism called on ALT or a "select", where a process may wait for a number of (input) channels. A channel may be with data, without data, or timeouts. In my world (opposite to Linux select), there is exactly one cause that may trigger the SELECT. I will use capitalized "SELECT" here to cover any such idiom, to differentiate from the Linux "select". The first cause that triggers the SELECT will be stored and cause a later rescheduling, but any potential causes that "lost", will not be allowed to cause an additional triggering. Only one cause per scheduling of a process. Very different from Linux select.

Said differently: the SELECT will put all the channels in the set to "first was here". When the first "second" contender on a channel in the set arrives, all the "first" attributes are removed from the rest of the set. This is a beautiful algorithm, that I think was first implemented in microcode on the first transputer processer in 1984. Inmos in Bristol, UK, made it, and David May and Tony Hoare (the father of CSP, "Communicating Sequential Processes" process algebra) were the main architects. The transputer basically was a machine designed to run the subset of CSP that was called occam. It's 28 years ago. Ada based its concurrency model on CSP. And 26 years after occam, Google designed the Go programming language, which also has based its concurrency model on CSP, just like the same guys had done at Lucent some 10 years before with their Limbo language. Except, at that time they didn't state the roots of their "chan of protocol". And transputer designer David May now is active with a company called XMOS, and a language called XC, based on the same ideas. So, CSP-based concurrency seems to be increasingly coming.

With this scheme communication equals synchronization.

In CSP they talk about "choice". It may be deterministic or "external" or nondeterministic or "internal" (often written as [] and |~|).

The SELECT is for the teacher who assigns work to his students. He then falls to sleep (or does other things) and is awaked when the first student is finished. The student is then served, and the SELECT is entered again. The new set may include the first student, when some fair scheme is used to avoid him spam the teacher. Or the SELECT may be non-deterministic. Read in the literature, f.ex. the Promela Wiki-article (more below).

With this scheme, the teacher may say that one of the students is not allowed to deliver his result before after two hours. So, there may be timers in the SELECT set. This would also make busy polling possible, should this be needed at the application level. But the run-time scheduler NEVER needs busy-polling. Another way to say this is that a process always has a cause when it's scheduled. Just think about Java's notify and notifyall, which may schedule a process for no reason!

Also, there may be an INPUT GUARD for each channel (applied to our special student for two hours). This is usually implemented as a conditional to each member of the SELECT set. Think of the SELECT GUARD set as a bit-mask, one bit per element in the set, just like in Linux select.

Now I'm getting closer to the theme of this note: Output guard vs. "channel ready" channel.

Some times it's required to buffer a message on a channel. In occam we used to do this by inserting a single buffer process, or a couple of processes to make one composite process called an "overflow buffer", which made it possible to detect a too eager producer and associated too slow consumer - at this application level. One time I needed a buffer of a hundred buffer positions, and I just started 100 buffers. Easy! The downside was that there would be 101 memory copies, so most often a ring buffer was administered inside the first of the two processes in the composite overflow buffer process.

The alternative could be for the eager producer (it has no idea itself how eager it is) to have something called an OUTPUT GUARD available. If you want to see it, have a look at the Promela process meta language (Wikipedia). This way, both inputs and outputs may be collected in the SELECT. So, having just one channel, plus an "else", then the potential receiver, if it is able to receive, will inform through the output guard if it's ok to send. If not, the else will be taken. The SELECT with both input and output looks like the Linux select, but that one doesn't have the synchronized mechanism in the bottom.

There is a code examples in [6] (here, search for bufchansched).

It's a nice mechanism. I have never had it in my world. It was not implemented on the transputer, since it was a multi-core programming language. And implementing it across a network (links) presumably was not worth it. There was another mechanism to get the same functionality. If you know what mechanism, please comment.

An output guard mechanism is good for detecting a broken line. To detected a connected line, there would have to be polling. Busy polling is most often necessary for this (assuming no hw edge interrupt). But the output guard seems not so relevant to handle the fast producer, slow consumer scenario - between internal processes. The same broken / connected scenario also would go for detecting that the other process has stopped, for some reason. Perhaps enough for a new blog note here.

So, here's the solution, to avoid the output guard, to avoid busy polling. The solution that also almost exists in Linux.

A standard channel has zero buffers. Have a look at Promela and Go. They both support buffered channels. So, why not add them to [1]? What should happen when such a channel is full? The channel send just returns a flag saying that it’s full, and the channel would not block in that situation. Observe that blocking is perfectly ok and does not impact what a system is able to do. With enough "parallel slackness" to get all the I/O going at required speed, having a process block (waiting in a non-busy way, for the second to arrive at the channel). But for a driver, close to the "asynchronous world" it's nice not to block to get rid of incoming messages. And it's nice not to send it into an asynchronous message system that may plainly overflow, crash and restart. We need a better system. We need the driver process to not block in that situation and itself handle overflows.

So, in the suggestion, after the return of the buffer full flag, the process knows this and if any incoming message arrives from the external asynchronous world, it may set a hold line, send a hold message or throw away the message - or whatever it decides to. It's up to the process.

In this state, the process will SELECT around a timeout channel and this channel that's triggered by the run-time system, that arrives when there is room in the channel. The receiver has picked out one message or all, that semantics is not so important here.

When this "channel ready" channel fires, the process can send off the last message, which may also be tagged with an overflow bit, on the channel. And it's guaranteed to succeed (unlike in Linux).

I was rather satisfied by inventing this on the plane home. I didn't like that it was already in Linux write and select. I have tried to describe some of the dissimilarities. See [2] for write, EAGAIN or EWOULDBLOCK and [3] for select (or pselect) and fd_set *writefds. It's in the context of writing to a Linux pipe.

But please, could anyone help me with describing how my solution is much "thread safer" than Linux? I need more than this:

I have more of a language, since I would do as described and reasoned above. Linux calls and Linux processes and threads are not understood by C or C++. Up to C++11 at least. And only so much understood by Linux. A C++ library for Linux needed several mutexes and semaphors to implement an ALT/SELECT in the CSP context [4]. So, are write and select usages in Linux really thread safe? I have a scheduler that is driven by the channels, and it's not preemptive. And why do Linux programmers brag that they "program single-threaded"? Is it an indication that the Linux process model is too coarse or difficult or expensive? So that they haven't learned to decompose into processes, when there is state or roles to hide away into a separate process? After object-orientation or OO, it's time for process-orientation as an additional idiom.

Should have been fun to set up a comparison table of the CSP type SELECT with channel ready channel and the similar Linux solution. Even if they are both fruits, I fear that one is apple and the other bananas?

A Linux pipe vs. channel trait

Observe that a Linux pipe is byte-wise. You send so many bytes and then so many bytes, and they are concatenated in the pipe. There has to be a common view of how to pick out each message or token.

This implies that you could read past a token, since a sender may have to break up a message into several chunks. There is no foldback, you cannot push anything back again. So, the receiver has to keep track of these things.

A channel is message based. A full message per chunk. Finito.

Common usage of Linux select

A collegue told me that the common usage of a Linux select is to remove active polling for events from remote machines from the application level, in say a http server (router). He said that he always used select to wait for events on already open sockets. One socket per bit in the select bit map.

Internally between threads he never had used select.

And he rarely used ip addresses internally between processes, so no select there either.

"My idea" - again

I found an interesting sentence in [5]. Here is its process-oriented specification of a buffer: If a buffer is empty it must be ready for input. If it contains some messages, then the buffer must be ready to output the next message required. It is also possible that it will accept further input, but it does not have to. (my comment: is this buffer full?) But here is the interesting sentence: "If further events are to be possible (such as a channel which can report on whether or not the channel is empty), then...". This is "my" channel! I am even more convinced now, that "my" idea was quite good!

Epilogue: "The XCHAN paper"

During June and July 2012 I had the opportunity to channel this blog note into a full blown paper for the CPA-2012 (at University of Abertay Dundee, Scotland 26-29 August 2012). I have published for the WoTUG and CPA series conferences several times before, but this time it was especially exciting. See [8] and here:

XCHANs: Notes on a New Channel Type
Øyvind TEIG, Autronica Fire and Security AS, Trondheim, Norway

Abstract. This paper proposes a new channel type, XCHAN, for communicating messages between a sender and receiver. Sending on an XCHAN is asynchronous, with the sending process informed as to its success. XCHANs may be buffered, in which case a successful send means the message has got into the buffer. A successful send to an unbuffered XCHAN means the receiving process has the message. In either case, a failed send means the message has been discarded. If sending on an XCHAN fails, a built-in feedback channel (the x-channel, which has conventional channel semantics) will signal to the sender when the channel is ready for input (i.e., the next send will succeed). This x-channel may be used in a select or ALT by the sender side (only input guards are needed), so that the sender may passively wait for this notification whilst servicing other events. When the x-channel signal is taken, the sender should send as soon as possible -- but it is free to send something other than the message originally attempted (e.g. some freshly arrived data). The paper compares the use of XCHAN with the use of output guards in select/ALT statements. XCHAN usage should follow a design pattern, which is also described. Since the XCHAN never blocks, its use contributes towards deadlock- avoidance. The XCHAN offers one solution to the problem of overflow handling associated with a fast producer and slow consumer in message passing systems. The claim is that availability of XCHANs for channel based systems gives the designer and programmer another means to simplify and increase quality.

References

[1] - https://www.teigfam.net/oyvind/pub/pub_details.html#NewALT
"New ALT for Application Timers and Synchronisation Point Scheduling"
Øyvind Teig and Per Johan Vannebo
Communicating Process Architectures 2009 (CPA-2009)
Peter H. Welch et. al. (Eds.)
IOS Press, 2009, ISBN 978-1-60750-065-0

[2] - http://linux.die.net/man/2/write

[3] - http://linux.die.net/man/2/select

[4] - Search for "C++CSP2"

[5] - Concurrent and real-time systems. The CSP approach. Steve Schneider. Wiley, 2000. Page 210, example about buffers (7.4.1).

[6] - Lecture at NTNU, Trondheim in April 2012: https://www.teigfam.net/oyvind/pub/NTNU_2012/foredrag.pdf - search for "bufchansched", since it's in Norwegian

[7] - Communicating Process Architectures 2012 (CPA-2012): http://www.wotug.org/cpa2012/. Full programme: http://www.wotug.org/cpa2012/programme.shtml

[8] - "XCHANs: Notes on a New Channel Type"

Øyvind Teig

Communicating Process Architectures 2012.

Peter H. Welch et. al. (Eds.)

ISBN 978-0-9565409-5-9

Read at https://www.teigfam.net/oyvind/pub/pub_details.html#XCHAN

.
.

030 - Burning an mp3 music CD from iTunes

First written 22.Aug.2011, updated 24.Aug.11. iTunes 10.4, Mac OS Snow Leopard 10.6.8.

Two entries for some songs, and a repair action

First:  The normal no-fuzz procedure is described at http://www.wikihow.com/Burn-iTunes-Music-to-MP3-CD-for-Stereos-and-CD-Players

I burnt an mp3 music CD the other day. Since some of the songs were not in mp3 format, I had to manually convert them to mp3 before I burnt the playlist. And I had to export the playlist to a file and then edit it in a text editor to replace all occurances of .m4a with .mp3. Then I had to import the playlist-file again. Only then did iTunes obey my wish to burn only .mp3 files, and not complain that "81 songs", sorry, could not be burnt.

When I afterwards looked at one of the original CDs in the Music folder, it had two entries for each song: one m4a and one mp3, or one mp3 and another list entry also as mp3! And playing that album, every song was played twice.

I then used Finder to track the music folder and move the mp3 files, that really should not be there, after the mp3 CD had been burnt, into another directory outside iTunes. Then I in iTunes scrolled through the list in the album, and it showed an exclamation mark for every song that was gone. Then I in iTunes picked them all out and moved them to the bin. The figure above shows the an album in the Music folder (left) and the almost same album as seen in the playlist.

mp3 CD in 2011?

Everybody else made mp3 CDs ten years ago. But it's only now that we in fact have two units that play them! The car stereo and the Denon CEOL. The second doesn't need mp3 CDs since it streams. And the car radio also has the aux input that takes music from iPad or iPhone's headphone outputs (see blog 028). But a few mp3 CDs in the car doesn't hurt, and makes it possible to see the song names in the car's display, and control music playing by steering wheel buttons. And since there's no USB memory stick input that would have obsoleted the mp3 CD, we will have to use standard CD, mp3 CD or aux input.

I wanted to extract the mp3 CD from our iTunes music collection.

Some more points

Some mp3 songs also all of a sudden appeared twice in the iTunes list, pointing to the same mp3 song. One lits entry has to go, so delete it, but do answer "yes" to the question whether you want to keep the file. I didn't realize and did kill a song or two. Some day I'll learn when I try to play it. Then it's going to be good with the synched backup I have!

If an album comes up with exclamation marks just by itself, it's only the list entry that's been added. Delete the exclamation mark lines only, since the mp3 file in itself is not repeated. Two files can't have the same name in the same directory.

Don't they want me to make mp3 CDs?

Probably not! Am I not supposed to make mp3 of purchased music? I think it's only ripped CDs that may be stored as mp3 out of the box, provided I had imported them to mp3.

I still haven't finished the clean-up! Some 40 albums take time! I really feel stupid.

Advice

Don't convert to mp3 if you want to burn an mp3 CD. Take the onces it has for you. Be sad and tell Apple.

But is there another way?

If there is something really wrong with what I did wrong, please comment! Like, if it's so wrong and it would have been easy to get it right, had I hit the right action trail immediately. Right?

Reported to http://www.apple.com/feedback/itunesapp.html.
.

029 - From ceramic vase to cutlery drainer

Moved to http://oyvteig-2.blogspot.com/2011/09/029-from-ceramic-vase-to-cutlery.html
.

028 - DIY prototype of an iPad 2 holder for car glove compartment

Moved to http://oyvteig-2.blogspot.com/2011/09/028-diy-prototype-of-ipad-2-holder-for.html
.

027 - Experiencing Apple AirPlay

This note is updated at my new blog space only, blog note http://www.teigfam.net/oyvind/home/technology/050-sound-on-sound-and-picture/. Welcome there!

Updated 27Sept2012 (after iOS 6 update to AppleTV), chapter "Playing from and to different devices"

Referring to blog 019 - we ended up with a Denon RCD N7 Network CD Receiver (CEOL). I gave up on Sonos. I got real AirPlay instead. Now we can listen to music at places and by combinations I hadn't read myself to know. Let me go on from blog 019:

Denon RCD N7 (CEOL)

This was bought at Hi-Fi klubben here in Trondheim in July 2011. Bringing it home, the first non-power cable I connected was the home network's ethernet cable. Joy! The only cables I plugged out from the 20 years old Denon Precision Audio Component / AM-FM Stereo Receiver DRA-335R were analogue. It has been tired some years now, together with the Denon PCM Audio Technology / Compact Disc Player DCD-680. Both were also bought at HiFiKlubben back then, and will now be stacked on the loft. Hard to scrap a friend.

The Ceol took charge and asked me if I wanted to update its firmware. Yes, I did - and it did the job in some 30 minutes.

I had bought the thing for its AirPlay capability (there are plenty other nice boxes without), and went to http://www.denon.eu/airplay/en/. First I had to pick out our unit's serial number, MAC address and upgrade ID. Paying with PayPal has always been a pleasure, so I expected the Ceol to be my everything's remote speaker any minute. For 48€ I thought it should. It didn't. Even if I got a receipt that all was fine - and in the background: Apple had got their licence fee. Since I get a salary every month I don't protest if others want some, even if they put it in the bank.

The Ceol told me I could upgrade with AirPort, ok. I pushed the right menu entries, and it said Authenticating for a minute, an hour, a night. All it accepts then is a power off, so I retried. Nope. A letter to HiFiKlubben and a reply from support adviced me to restart to factory settings. The updated firmware and the 48€ investment should be fine. In the manual it's called "Resetting the microprocessor: unplug the power cord, then hold volume up and down simultaneously and plug in the cable, then release the buttons". However, I used the slightly different procedure in the support email: "Total reset (to factory settings out of the box / virgin mode) 1. Switch Off the unit and unplug mains plug from wall outlet. 2. Press and hold both Volume up and Cursor down buttons on the front panel and plug the mains plug into the wall outlet. 3. Release the two buttons."

This worked perfectly (as probably would have the manual's recipe?): AirPlay soon was in the box. And everywhere! Running Apple's Remote app on the iPad now, the iTunes library comes up as usual, but both the Ceol and the computer come up! I get "surround sound" that I did't know. The tv speakers with nice bass sound (from JackOSX router on the Mac Mini, see blog 019) also appeared. And they are in phase (thanks to some clock synchronization messages in the AirPlay protocol I believe).

Now Gary Moore's Ballads & Blues 1982-1994 plays like music has never been heard in this house. At the moment: bliss.

Experience

After a month's use I'd say I'm pretty satisfied with AirPlay. But there are some repeating weak spots:

1. Sound level at connection.
   I still haven't figured out why it comes on with an enormous volume some times. So, if the amp is more powerful than the speakers, the AirPlay start-up volume may destroy your speakers! For me, it's ok, though.
2. Sound level up when down or down when up
   The app volume control and the real volume may be out of synch before I have touch the app's volume control. So, when I try to push the volume down (from a shown high volume), it increases (from the unit's low volume) - so that the overall effect is increased volume! Aften that up is up and down is down.
3. Broken connection.
   Some times one of the AirPlay "speakers" fall out, and I'd have to manually reconnect.
4. Speakers out of phase.
   One time, while iTunes (version 10.4) converted a bunch of songs to mp3 (and was very busy in one of its "threads"), and it was playing a song simultaneously - the speakers went out of synch. I know this, since it was easy to hear. I would guess 50 ms since it was not that much.
5. "Connecting to AirPlay unit" when envisaged as already connected.
   Every time I find a new melody it seems to disconnect and then needs some extra time to reconnect. I experience this as not necessary.
6. Outdated display.
   I have seen Denon not display the present melody, when I played iTunes through the Remote app. Didn't iTunes send it, or didn't Denon update? It stayed with the other day's melody through a whole playing session. At the moment I dont't know what updates it.

Reported to http://www.apple.com/feedback/itunesapp.html. I have found no similar feedback link to Denon.

Playing from and to different devices

We finally (in July 2012) bought an Apple TV unit. This chapter does not discuss the Apple TV per se, but how AirPlay works . The figure above does not show all details, and I have done some shortcuts. Have a look at note 019 for more details (but not updated with Apple TV).

I am most concerned with when the sound is synchronized (playing without relative delays between individual units) in this note. Mirroring the screen is only possible to one screen, I assume. Anyhow, I can listen to sound from many speakers (connected to several amplifiers) but only want to see one screen at a time. The iPad and iPhone (as well as OSX Mountain Lion, which I don't have) can mirror the screen over AirPlay. That is just so nice.

Is the unwanted delay caused by the HDMI? The screen is a Samsung RE26E0 TV (from 2005-06). The unwanted delays I am describing here may have to do with the internal D/A-conversion delay for the HDMI cable. But then, shouldn't that also have given reckognisable delay for the HDMI from the TV tuner? See discussion thread "AirPlay audio sync issue" here (thanks, Anders!). That thread also mentions that AirPlay has a buffer delay of 2 seconds, so there should be no delays between the immediate outputs from any AirPlay unit. If I only had an optical input D/A converter to test the Apple TV's direct output.

YES! It helped to remove the default value of the programmed delay in the Samsung TV! Here's the help that Anders (again) dug out. See post 5 in there as to how to start the service menu. Then the Samsung RE26E0 TV menu was like this (the TV runs as normal with picture and sound):

• From post 5: "Turn off the TV then press Info -> Menu -> Mute in sequence then turn the TV back on again. This will get you into the secret Service Menu." (I had to fiddle a little to get it into that mode)
• Ok, here's the top of the secret Service Menu:
• Select "6. STV8257/STA323W"
• I set "HDMI Delay" from "41H" to "00H" (41 Hex is (4*16)+(1*1)=65)
• I also set "PC Delay" from "41H" to "00H"
• There was no "save config", so it was saved immediately

Then the delay through the HDMI port was zero! AirPlay has always been working!

CASE 1: Playing sound from Apple TV

1. Since I don't use the optical sound output from the Apple TV, the TV has to be on to get the sound to any speakers from the Apple TV. This is no problem, we need its screen anyhow.
2. (Obsoleted: With sound from Apple TV, only the green (left) speakers may get sound. Not the two others!)
   New with iOS 6 upgrade on AppleTV (late Sept. 2012): Now it is possible to send AirPlay music to the other AirPlay units! It's also possible to adjust the volume with the AppleTV remote control on the AppleTV.
3. The green speakers are volume adjusted with the TV remote control. Apple TV has no volume control (not true for AirPlay outputs with iOS 6, see last sentence in bullet above.)
4. Observe the difference between playing from Apple TV and playing with Apple TV. More below.

CASE 2: Playing sound with Apple TV

1. As I mentioned in the topmost chapter, playing from the Mac Mini (with iTunes) into the blue speakers (center) and the black Denon speakers (right) are completely synchronous. The Mini runs Lion, not Mountain Lion. The latter can mirror its screen to the Apple TV. Our Mini is too old for that.
2. If the following happens to you then try to check the HDMI delay, see above, to see if you can get rid of it. (I try to leave traces of the path to a solution in my blogs, and this is a paragraph "on the road":) "However, adding the Apple TV as the third speaker is not useful, since there is a reckognisable delay. It's not at all in synch! Routing the sound to Apple TV in this case does not make sense."

CASE 3: Playing from iPad or iPhone and Mini

1. When selecting AirPlay units from iPad or iPhone, the Mini output (blue, center) does not appear.
2. This is easy to understand, since Lion is not a first class citizen of AirPlay.
3. When I run iTunes on the Mini, however, "This computer" appears as a selection - in addition to the other two.

.

.

026 - iOS picture sorting and file system do not tango

(2 July 2011, updated 24 October 2011)

This note discusses synching pictures on iOS units (iDevices) by letting iTunes use a disk directory for picture synching. It does not initially deal with having iTunes synch using iPhoto, but I plan to also handle that. I use Mac OS X Snow Leopard. I will update this note as time goes. If it's not been updated, it's probably because I haven't seen anything new.

iOS sorting by modification date only

Update 24.Oct.2011 with iOS 5.0: Apple seem to have gone back to alphabetical sorting again! However, in view of what I have written below, maybe they now have combined sorting methods, as I have suggested? 

Apple has decided that the iPad, iPhone or iPod (all running iOS) pictures and movies shall be sorted according to last modification date and time. It didn't start that way, but has been like that since iOS 4.

I would perhaps want iOS to let me select sorting method. Letting me choose alphabetical sorting could have been great, provided it worked. Since the alphabet I use is sortable (!?), it's a nice feature that our culture has become accustomed to. However, since camera picture names contain a number, the series would sooner or later wrap around: DSC_9999 would become DSC_0000. So any number would repeat, making alphabetical sorting difficult. And no matter how much time increases, it only seems to wrap at each big bang(?). (Being an embedded programmer I know that having a finite word length, counting the number of system clock tics does indeed overflow to zero. But that's why we're there, to know about these things.)

Maybe combining some date/time and alphabetical sorting would have been smart? The engineers at Apple of course know that being smart some times is not always that smart. The framework in which to be smart may change, and in the new context it wasn't smart any more.

So, sorting photos by date/time has come to stay. I'll have to take it in.

Vacation pictures oops.. changed!

If I change a file, cropping it, resizing it, change the light or anything, then it would now cause that picture to get today's modification date/time. Synching it with iTunes to an iOS unit would place it down at the bottom in the sorted list.

So I need to overwrite modification date by creation date

I found an AppleScript that does exactly this on the net [1]. I'll call it "TheScript" from now on. Perfect, now the photos are in lane again after the next iTunes synch also. But hey, finding it, downloading it and start to use it is not simple in any context, Apple?

Say the disk directory where iTunes finds the pictures is on a local disk, and say that TimeMachine is used for backup, then all is fine.

However, if the picture directory is on a network disk, or if as above without TimeMachine as backup, or if not only backup but also synchronization is used, then I may have a problem.

My synch program may push my pictures out of lane again!

Synching disks and synching iOS units with  iTunes are not the same thing. I'll try to be precise below. Later in 2011 Apple iCloud synching will happen without iTunes.

Skip this chapter if you always plan to run TheScript before you synchronize the disk with another disk, after having modified a file.

To get disk synchronization to work, modifying a set of files should be "atomic" with respect to disk synching. So, make yourself non-disturbable in that phase. A modified file should not be synched before TheScript!

I'll try to explain the situation that might arise if you modified a file, got disturbed, and then synched the disks "out of synch".

I use an old friend called ChronoSync. I have used it for years to synch several machines, using a network disk as the base. Whenever I change a photo, I do it at some local machine and synch the disks and then let iTunes use the synched base at the network disk for its photo synching.

However, since I have modified a photo and thus made it newer, ChronoSync will (if I run it in this phase) let the newest version survive. Later iTunes iOS synching will place the picture at the bottom.

I have found no way for the present ChronoSync to work this out. A change could be something like this, to be used after file change and we don't have TheScript:

Resolve conflict if two files have equal creation date to use the newest modified file but then change modification dates of both to become creation date

Or, how about a special synch session called iOS picture/movie synching?

I have informed Econ Technologies about this 2July11 (http://www.econtechnologies.com//pages/support/support_comment.php). (And they have replied, stay tuned)

Back to iTunes: re-date alphanumerically sorted files?

I have no way to solve this! The name of all my files have started with som number and then a sentence, like "314 pythagoras.jpg" and "271 e.jpg". The operating system understands that the characters '0'-'9' are treated as numbers and sorted accordingly. I have called this alphabetical sorting, even if it does understand fields. So the 271 file is sorted as the first.

Some times I could learn AppleScript and write a script that solves my problem: keep dates as much as possible, understand modification date and spread the set out over the dates wanted. Sorting by pushing around pictures by hand is perhaps best.. But this is difficult!

Inserting a picture or a movie into the iOS sorting sequence

Some times you need to add a picture into a sequence. Maybe you want a poster in front of each section, or the next country's flag. This is almost easy: 1.) Make a copy of the neighboring file. This has the same creation date. Experiment with the file before or after, it's not obvious how equal date/times are sorted. 2.) Open the new file in an editor. I use Graphic Convertor. 3.) Paste the image you have made prior to this so that it's inserted into the just opened picture, so that all is overwritten. You may have to delete all contents first. 4.) Save the file with the new contents. 5.) Run TheScript to make modification date from creation date.

This works for picture files.

Movies like .mov I have not been able to fix yet. Stay tuned.

Refs

[1] - MacFileDatesChanger by Daniel A. Shockley, http://www.danshockley.com. Includes some code from Paul Berkowitz and Nigel Garvey on applescript-users mailing list.

.