Converting podcasts to regular tracks in iTunes

I have spent the better part of the morning trying to work out how to convert podcasts to "regular" tracks in iTunes, so they would show up in shuffle, etc. Mostly this was for my collection of Jonathan Coulton "Thing a Week" episodes, but it would be useful for anything you wanted to move out of your podcast list into the main audio portion of the library. I suppose the reason it took so long to find the solution is I started by searching for it on Google instead of just looking through the iTunes menu options, though as you will see the solution wasn't immediately obvious even once I had found it.

I knew that iTunes would let me change settings like "Remember playback position" and "Skip when shuffling" from the info dialog (Cmd-I), so I started there. The only setting that even mentioned podcast was the genre, and I already knew that was not what I needed to change. Some of the tracks already had "real" genres and only some were set to "Podcast".

I also knew there is a separate podcast flag available for queries in Smart Lists and AppleScript, since I used it to create my "Active Queue" podcast playlist with selections of episodes from various podcast series (it's like creating your own mix tape, but for talk radio). I tried to write a simple AppleScript to change the podcast flag of selected tracks to true. It turns out the flag is a read-only attribute. No amount of searching uncovered any way to change the setting using AppleScript.

The first useful looking suggestions I ran into were to convert the ID3 tag format to an older version, then convert it back. Doing that erased most of the comments and other meta-data associated with the tracks, though, so I didn't like the results.

Next I found a few forum and blog posts that talked about an ITUNESPODCAST setting in the extended ID3 tags. They all mentioned a Windows program for removing or changing the flag, though. I examined a few of the files with Ned Batchelder's python module id3reader, but didn't see anything that looked like "ITUNESPODCAST" in the output.

Going back to Google, I finally found a reference to converting the files to AAC using an option in the Advanced menu. That seemed like overkill, but at this point I was becoming fed up and just wanted to be done with the whole thing. I could always re-encode as MP3, after all. Well, iTunes didn't have a menu option to "Encode as AAC". It did have "Convert selection to MP3", which didn't make much sense to me. As far as I knew, the tracks were already MP3 files. But lo and behold, selecting that menu option did enable them in the iTunes Music Library. It made copies of all of the tracks as it converted them, so I could even delete the podcast subscription.

So, if you want to add podcast episodes you have already downloaded to your music library and turn off the podcast flag, select the track and choose Advanced->Convert selection to MP3.

PyMOTW: Queue

Module: Queue
Purpose: Provides a thread-safe FIFO implementation
Python Version: at least 1.4

Description:

The Queue module provides a FIFO implementation suitable for multi-threaded programming. It can be used to pass messages or other data between producer and consumer threads safely. Locking is handled for the caller, so it is simple to have as many threads as you want working with the same Queue instance. A Queue's size (number of elements) may be restricted to throttle memory usage or processing.

This disucssion assumes you already understand the general nature of a queue. If you don't, you may want to read these references before continuing:

• Queue data structures
• FIFO

Example:

As an example of how to use the Queue class with multiple threads, we can create a very simplistic podcasting client. This client reads one or more RSS feeds, queues up the enclosures for download, and processes several downloads in parallel using threads. It is extremely simplistic and is entirely unsuitable for actual use, but the skeleton implementation gives us enough code to work with to provide an example of using the Queue module.

To start, we import a few useful modules:

from Queue import Queue

from threading import Thread
import time

import feedparser

Now, let's establish some operating parameters. Normally these would come from user inputs (preferences, a database, whatever). For our example we hard code a few values:

# Set up some global variables
num_fetch_threads = 2
enclosure_queue = Queue()

# A real app wouldn't use hard-coded data...
feed_urls = [ 'http://www.castsampler.com/cast/feed/rss/guest',
            ]

Next, we need to define the function that will run in the worker thread, processing the downloads. Again, for illustration purposes this only simulates the download. To actually download the enclosure, check out the urllib module, which we will cover in a later episode. In our example, we sleep a variable amount of time, depending on the thread id.

def downloadEnclosures(i, q):
   """This is the worker thread function.
    It processes items in the queue one after

    another.  These daemon threads go into an
    infinite loop, and only exit when
    the main thread ends.
    """
   while True:

       print '%s: Looking for the next enclosure' % i
       url = q.get()

       print '%s: Downloading:' % i, url
       time.sleep(i + 2) # instead of really downloading the URL, we just pretend

       q.task_done()

Once this target function is defined, we can start the worker threads. Notice that downloadEnclosures() will block on the statement "url = q.get()" until the queue has something to return, so it is safe to start the threads before there is anything in the queue.

# Set up some threads to fetch the enclosures
for i in range(num_fetch_threads):

worker = Thread(target=downloadEnclosures, args=(i, enclosure_queue,))

worker.setDaemon(True)
worker.start()

And now we retrieve the feed contents (using Mark Pilgrim's feedparser module) and enqueue the URLs of the enclosures. As soon as the first URL is added to the queue, one of the worker threads should pick it up and start downloading it. The loop below will continue to add items until the feed is exhausted, and the worker threads will take turns dequeuing URLs to download them.

# Download the feed(s) and put the enclosure URLs into

# the queue.
for url in feed_urls:
   response = feedparser.parse(url, agent='fetch_podcasts.py')

   for entry in response['entries']:
 for enclosure in entry.get('enclosures', []):

     print 'Queuing:', enclosure['url']
     enclosure_queue.put(enclosure['url'])

And the only thing left to do is wait for the queue to empty out again.

# Now wait for the queue to be empty, indicating that we have
# processed all of the downloads.
print '*** Main thread waiting'
enclosure_queue.join()
print '*** Done'

If you download the sample script and run it without modification, you should see output something like this:

0: Looking for the next enclosure
1: Looking for the next enclosure
Queuing: http://http.earthcache.net/htc-01.media.globix.net/COMP009996MOD1/Danny_Meyer.mp3
Queuing: http://feeds.feedburner.com/~r/drmoldawer/~5/104445110/moldawerinthemorning_show34_032607.mp3
Queuing: http://www.podtrac.com/pts/redirect.mp3/twit.cachefly.net/MBW-036.mp3
Queuing: http://media1.podtech.net/media/2007/04/PID_010848/Podtech_calacaniscast22_ipod.mp4
Queuing: http://media1.podtech.net/media/2007/03/PID_010592/Podtech_SXSW_KentBrewster_ipod.mp4
Queuing: http://media1.podtech.net/media/2007/02/PID_010171/Podtech_IDM_ChrisOBrien2.mp3
Queuing: http://feeds.feedburner.com/~r/drmoldawer/~5/96188661/moldawerinthemorning_show30_022607.mp3
*** Main thread waiting
0: Downloading: http://http.earthcache.net/htc-01.media.globix.net/COMP009996MOD1/Danny_Meyer.mp3
1: Downloading: http://feeds.feedburner.com/~r/drmoldawer/~5/104445110/moldawerinthemorning_show34_032607.mp3
0: Looking for the next enclosure
0: Downloading: http://www.podtrac.com/pts/redirect.mp3/twit.cachefly.net/MBW-036.mp3
1: Looking for the next enclosure
1: Downloading: http://media1.podtech.net/media/2007/04/PID_010848/Podtech_calacaniscast22_ipod.mp4
0: Looking for the next enclosure
0: Downloading: http://media1.podtech.net/media/2007/03/PID_010592/Podtech_SXSW_KentBrewster_ipod.mp4
0: Looking for the next enclosure
0: Downloading: http://media1.podtech.net/media/2007/02/PID_010171/Podtech_IDM_ChrisOBrien2.mp3
1: Looking for the next enclosure
1: Downloading: http://feeds.feedburner.com/~r/drmoldawer/~5/96188661/moldawerinthemorning_show30_022607.mp3
0: Looking for the next enclosure
1: Looking for the next enclosure
*** Done

YMMV, depending on whether anyone modifies the subscriptions in the guest account on CastSampler.com.

References:

Python Module of the Week


Updated 5/20/2007 with technorati tags.
Updated 9/5/2007 with minor formatting fixes.

Technorati Tags:
python, PyMOTW

Things to Do

In no particular order:

1. Cull my Google Reader subscriptions. 364 is too many.
2. Finish reading Dreaming in Code.
3. Add tagging support to codehosting.
4. Verify all of the domains under my control with Google Web Master tools.
5. Create a Trac plugin for code reviews based on the process we use at work.
6. Change the monitor feeds on CastSampler.com so they do not include items without enclosures.
7. Enhance BlogBackup to save enclosures and images linked from blog posts.
8. Write a tool to convert an m3u file to an RSS/Atom feed for Patrick so he will set up a podcast of his demo recordings.
9. Improve AppleScript support in Adium.
10. Add support to Adium for notifications when a screen name appears in a chat message.
    

CastSampler.com monitoring feeds

On the plane back from Phoenix this week, I implemented some changes to the way CastSampler.com republishes feeds for the sites a user subscribes to. The user page used to link directly to the original feed so it would be easy to copy it to a regular RSS reader to keep up to date on new shows. That link has been replaced with a "monitor" feed which uses the original description and title for each item, but replaces the link with a new URL that causes the show to be added to your CastSampler queue. The user page still links to the original home page for the feed, so I think I am doing enough as far as attribution and advertisement. Any author information included in the original feed is also passed through to the monitor feed. The OPML file generated for a user's feeds links to these "monitor" feeds instead of the original source, too.

The goal of these changes is to make it easy to use a feed-reader such as Bloglines or Google Reader to monitor podcasts from CastSampler. To add an episode to your queue, just click the link in the monitor feed to be directed to the appropriate CastSampler.com page.

By the way, how cool is it to be able to develop a web app on my Powerbook while I'm on a plane? What an age to be alive.

CastSampler.com

My most recent project is CastSampler.com, a tool for building a personal "mix-tape" style podcast. I tend to listen to one or two episodes from a lot of different shows, so I don't want to subscribe to the full show feed. Instead, I add the show to my CastSampler list, then I can add only those episodes that I want to my personal feed.

I have plenty of work left to do, but the basic features all work now so I would love to get some feedback.