[Ardour-Users] X Runs at Exactly 10 Minute Intervals
Brett Clark
Brett at ciscoinc.com
Thu Sep 27 06:34:30 PDT 2007
>> Do I need to recompile Qjackctl
Not Qjackctl because its just a GUI frontend, but you will need to recompile jackd with the '--with-default-tmpdir=/mnt/ramfs' option. This is something that should be done anyway, but i dont think it would be the only cause for your issue. If this was the only problem then you wouldnt be seeing XRUNS at exactly 10 minute intervals, they would be more spuratic.
The vmstat does show an increase in the 'bi' column (Blocks In/Sec). Im not sure if this column includes network traffic too.
The other thing that Florian mentioned is that you need (or really really should ) be running a realtime kernel. Can you please verify?
Thanks
--Brett
________________________________
From: Martin Lynch [mailto:martin.lynch at shaw.ca]
Sent: Wed 9/26/2007 10:50 PM
To: Brett Clark; ardour-users at lists.ardour.org
Subject: Re: [Ardour-Users] X Runs at Exactly 10 Minute Intervals
Again, many thanks for taking the time, Brett - appreciate your suggestions!
Here's what your 'ps' command returned - are you saying you only see 5 entries? Wonder why mine shows 10!?! Anyway, looks to me like 6 of them are in RT, with the 6th one showing the '89' I set in Qjackctl:
~$ ps -C jackd -L -o rtprio,pri,cmd
RTPRIO PRI CMD
- 16 /usr/bin/jackd -v -R -P89 -p128 -t1000 -dfreebob -dhw:0 -r96000 -p64
- 24 /usr/bin/jackd -v -R -P89 -p128 -t1000 -dfreebob -dhw:0 -r96000 -p64
- 24 /usr/bin/jackd -v -R -P89 -p128 -t1000 -dfreebob -dhw:0 -r96000 -p64
- 19 /usr/bin/jackd -v -R -P89 -p128 -t1000 -dfreebob -dhw:0 -r96000 -p64
99 139 /usr/bin/jackd -v -R -P89 -p128 -t1000 -dfreebob -dhw:0 -r96000 -p64
89 129 /usr/bin/jackd -v -R -P89 -p128 -t1000 -dfreebob -dhw:0 -r96000 -p64
92 132 /usr/bin/jackd -v -R -P89 -p128 -t1000 -dfreebob -dhw:0 -r96000 -p64
92 132 /usr/bin/jackd -v -R -P89 -p128 -t1000 -dfreebob -dhw:0 -r96000 -p64
99 139 /usr/bin/jackd -v -R -P89 -p128 -t1000 -dfreebob -dhw:0 -r96000 -p64
93 133 /usr/bin/jackd -v -R -P89 -p128 -t1000 -dfreebob -dhw:0 -r96000 -p64
1) No, haven't changed any priorities via 'setpci'.
2) I've added the following to fstab:
none /tmp/jack tmpfs defaults 0 0
none /mnt/ramfs tmpfs defaults 0 0
Does that seem right? Do I need to recompile Qjackctl with:
--with-default-tmpdir=/mnt/ramfs
to enable this? I'd like to make sure I've got this right before I reboot and all hell breaks loose...
3) Here are the vmstat 5 results (the X run occurred around the time where the "r" column shows 2 or 3):
0 0 109252 15632 2324 589840 0 0 3 1928 9838 26625 17 9 73 0
1 0 109252 16216 2248 589580 0 0 4 1947 9861 28769 20 8 72 0
0 0 109252 16864 2048 589092 0 0 3 1273 9792 25719 17 9 73 0
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
1 0 109252 16680 1924 589256 0 0 6 1598 9704 26152 20 14 66 1
1 0 109252 16744 1864 589112 0 0 3 1235 9770 25621 19 16 65 0
2 0 109252 16492 1772 589232 0 0 3 1765 8807 22820 20 14 66 0
3 1 109252 16152 1560 590092 0 0 21 1842 9849 26606 18 5 76 1
3 0 109252 15832 1588 590352 0 0 47 1501 9881 27767 21 8 70 0
4) Here are the iostat 5 results (X run occurred during either the 2nd or 3rd set):
avg-cpu: %user %nice %system %iowait %steal %idle
19.70 0.00 11.20 0.60 0.00 68.50
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 1.80 6.41 24.05 32 120
hda 0.00 0.00 0.00 0 0
sdb 4.41 4.81 1633.67 24 8152
sdc 3.61 4.81 1622.44 24 8096
md0 408.22 9.62 3256.11 48 16248
avg-cpu: %user %nice %system %iowait %steal %idle
20.60 0.00 13.20 0.20 0.00 66.00
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 0.80 0.00 5.20 0 26
hda 0.00 0.00 0.00 0 0
sdb 3.60 0.00 1768.00 0 8840
sdc 4.00 6.40 1787.20 32 8936
md0 445.20 6.40 3555.20 32 17776
avg-cpu: %user %nice %system %iowait %steal %idle
16.98 0.00 10.59 0.20 0.00 72.23
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 1.00 0.00 15.97 0 80
hda 0.00 0.00 0.00 0 0
sdb 3.59 1.60 1786.83 8 8952
sdc 4.19 4.79 1802.79 24 9032
md0 449.50 6.39 3589.62 32 17984
No obvious activity jumps out at me (but I'm still a noob...)
Brett Clark wrote:
Martin,
# ps -C jackd -cmL
As I understand it, the FF indicates realtime.
i want to point out that the CLS column in your 'ps -C jackd -cmL' command does not show priority. Its a 'class' column and FF shows FIFO. The following command will show you both the priority and the RT status of jackd and threads: ps -C jackd -L -o rtprio,pri,cmd
Im not sure how many of the threads should have RT, but i know at least 1 of them should. I actually show 5 threads on my system and only 2 of them have RTPRIO values. At the same time, im not sure i should make any claims that my system is setup perfect either :)
the column labeled PR = Priority?
Yes. But this is scheduler priority as opposed to the RT priority (RTPRIO).
Im grasping at straws with these questions:
1) have you changed any of the latencies on your PCI bus (setpci)?
2) Does jackd use a tmpfs mount as opposed to a regular drive? For instance, my jackd is set to use a RAM drive instead of a physical HDD to reduce overhead.
3) can you run a 'vmstat 5' around the 10 minute mark and see if the cpu 'us' or 'sy' spike? Also check for swapping.
4) if you run an 'iostat 5' at the 10 minute mark, do you see a spike in blk_read or blk_wrtn? You will want to ignore the first output and watch the additional outputs.
#3 and #4 are just my attempts to try and narrow down if its a cpu/mem thing or if its an hdd/io thing.
Disclaimer: Im actually a Redhat (Fedora) guy, so i may be off base with some of my assumptions and ideas.
Thanks
--Brett
________________________________
From: Martin Lynch [mailto:martin.lynch at shaw.ca]
Sent: Wed 9/26/2007 11:34 AM
To: Brett Clark
Subject: Re: [Ardour-Users] X Runs at Exactly 10 Minute Intervals
Many thanks for the response Brett!
I don't have a /var/log/cron file, however I did have a look at every file in /var/log and couldn't spot anything suspicious. Using top, the only thing that came to the top was Xorg, but it comes to the top on a fairly regular basis (many times a minute). Does the column labeled PR = Priority? If so, it's showing 15, which is odd since I have it set to 89 in Qjackctl. /etc/cron.d has 4 files, none of which appear to be scheduled at 10 min intervals.
I wondered about your polling suggestion also. The only app I can recall which might do this is Thunderbird, which I have set to check for mail every 10 mins (however I close it before using JACK). As an experiment I'll try changing it to 20 mins and see if that helps.
Any other thoughts or places I could look for 10 minute occurrences?
Thanks again - really appreciate you taking the time to help!
Brett Clark wrote:
This sounds very suspicious like a cron entry. You mentioned checking the System Manager, but im wondering if that would show all potential cron jobs. Can you do a 'tail /var/log/cron' and see if there's anything executing at the time of the XRuns. Usually there will be an sa1 audit running every 10 minutes, but that should be pretty light. Additional cron jobs that execute in that timeframe could be scheduled in the /etc/cron.d directory.
Another problem could be a constant running job that does some type of polling at 10 minute intervals and increases the system load. One way to locate this would be to run 'top' and see what pops to the top when the XRuns happen.
--Brett
________________________________
From: ardour-users-bounces at lists.ardour.org on behalf of Martin Lynch
Sent: Wed 9/26/2007 10:44 AM
To: ardour-users at lists.ardour.org
Subject: [Ardour-Users] X Runs at Exactly 10 Minute Intervals
Hoping someone can help me figure out what's causing X runs at precisely 10 minute intervals - not 9:59, not 10:01 but exactly 10 mins.
My setup:
AMD 64 X2
Feisty
2 GB DDR2 800 mhz
M-Audio Firewire Solo using Freebob
Sample rate: 96,000
Periods: 3
Frames: 64
Latency: 2 ms
Realtime checked in Qjackctl
Priority = 89
Qjackctl 0.2.21
# ps -C jackd -cmL
shows:
PID LWP CLS PRI TTY TIME CMD
24142 - - - ? 00:14:40 jackd
- 24142 TS 20 - 00:00:00 -
- 24143 TS 24 - 00:00:00 -
- 24144 TS 24 - 00:00:00 -
- 24145 TS 24 - 00:00:00 -
- 24146 FF 139 - 00:00:00 -
- 24147 FF 129 - 00:05:51 -
- 24148 FF 132 - 00:00:00 -
- 24149 FF 132 - 00:00:00 -
- 29990 FF 139 - 00:00:00 -
- 29991 FF 133 - 00:00:25 -
As I understand it, the FF indicates realtime.
Messages from JACK:
Load = 6.0202 max usecs: 44.000, spare = 622.000
LibFreeBoB ERR: SLAVE XMT : Buffer underrun! 64 (0 / 256) (0 / 0 )
17:34:51.873 XRUN callback (2).
FreeBoB MSG: xrun detected
LibFreeBoB ERR: Xrun on connection 1
load = 53.0101 max usecs: 3481.000, spare = 0.000
load = 29.7333 max usecs: 43.000, spare = 623.000
Load = 4.9502 max usecs: 25.000, spare = 641.000
LibFreeBoB ERR: MASTER RCV: Buffer overrun!
LibFreeBoB ERR: Xrun on connection 1
17:44:51.782 XRUN callback (3).
FreeBoB MSG: xrun detected
load = 4.4270 max usecs: 26.000, spare = 640.000
Note that in the first instance it's an underrun; 2nd instance an overrun, which seems odd to me (ie I'd think with the X runs happening on such a precise schedule it would show the same problem each time).
I've tried using System Manager to track down any processes which might be scheduled to run every 10 mins and can't pin down anything. I also tried stopping ntpd but the problem continued.
Other programs which run simultaneously:
- Beryl
- Mythtv backend
Using Ardour neither increases the X runs nor eliminates them - they still pop up at 10 min intervals.
Any ideas or suggestions? Could Freebob be the problem? Are there system processes known to run at 10 min intervals that could be interfering?
Also, not sure this has anything whatsoever to do with my X runs, but Qjackctl also reports a whole series of:
QTextCursor::gotoParagraph Index: 49 out of range
load = 4.1724 max usecs: 26.000, spare = 640.000
load = 4.0381 max usecs: 26.000, spare = 640.000
load = 3.9710 max usecs: 26.000, spare = 640.000
QTextCursor::gotoParagraph Index: 49 out of range
QTextCursor::gotoParagraph Index: 49 out of range
I also tried running JACK from console, with the same results.
Driving me nuts as every time I need to do a take, I have to wait until an X run hits, then I have a 10 min window to record, which leaves me thinking about the impending X run, rather than the part I'm playing!
Many thanks!
More information about the Ardour-Users
mailing list