Pipe question

13 posts / 0 new
Log in or register to post comments
Last post
Fri, 2015-04-17 12:21
#1
trixie
trixie's picture
Offline
Last seen: 2 days 15 hours ago
Joined: 2011-02-03 13:58
Pipe question
I have a program that uses a plugin system. The main executable feeds audio data to the selected plugin sequentially. Whenever a chunk of data is received, the plugin invokes an encoding routine, which then writes the encoded portion of data into a file. The result is an MP3, FLAC etc. file, depending on the selected plugin. I'd like to develop a new plugin that will not contain an encoding routine but, instead, will use an external encoder like Lame or ffmpeg to do the job. SoundFX (the old audio processing software) does that using a pipe which feeds data to Lame (for MP3) or Pegase (for MP2). I've never used a pipe before, and I'd be very grateful for any tips / suggestions / caveats / code examples concerning the implementation using the AmigaDOS PIPE: Thanks!
Top
Fri, 2015-04-17 16:20
broadblues
broadblues's picture
Offline
Last seen: 3 years 1 month ago
Joined: 2012-05-02 21:48
SoundFX (the old audio
SoundFX (the old audio processing software) does that using a pipe which feeds data to Lame (for MP3) or Pegase (for MP2).
Ugh how unix :-) Didn't realise it did that thought it had internal (and broken) encoders for those formats, might have to revue my SFX setup, as that could be useful... Heres is how abc-shell handles pipes. 
  1. int pipe(int filedes[2])
  2. {
  3.         char pipe_name[1024];
  4.  
  5. #ifdef USE_TEMPFILES
  6. #if defined(__amigaos4__)
  7.         snprintf(pipe_name, sizeof(pipe_name), "/T/%x.%08x",
  8.                  pipenum++,((struct Process*)FindTask(NULL))->pr_ProcessID);
  9. #else
  10.         snprintf(pipe_name, sizeof(pipe_name), "/T/%x.%08x",
  11.                  pipenum++,GetUniqueID());
  12. #endif
  13. #else
  14. #if defined(__amigaos4__)
  15.         snprintf(pipe_name, sizeof(pipe_name), "/PIPE/%x%lu/32768/0",
  16.                  pipenum++,((struct Process*)FindTask(NULL))->pr_ProcessID);
  17. #else
  18.         snprintf(pipe_name, sizeof(pipe_name), "/PIPE/%x%08x/4096/0",
  19.                  pipenum++,GetUniqueID());
  20. #endif
  21. #endif
  22.  
  23.         filedes[1] = open(pipe_name, O_WRONLY|O_CREAT);
  24.         filedes[0] = open(pipe_name, O_RDONLY);
  25.  
  26.     if (filedes[0] == -1 || filedes[1] == -1)
  27.         {
  28.                 if (filedes[0] != -1)
  29.                     close(filedes[0]);
  30.                 if (filedes[1] != -1)
  31.                     close(filedes[1]);
  32.  
  33.                 FUNCX;
  34.                 return -1;
  35.         }
  36.  
  37.         FUNCX;
  38.         return 0;
  39. }
The readonly handle above becomes the stdin of your subprocess, and the parent writes data to the write end. If you need to read data back then the the reverse is done, writeonly becomes stdout of subprocess and you read from the readonly end. It's important to open the write end first (at least I've had issues with pipes blocking when I didn't). That code is a bit of mess and you would most likely want to use DOS cxalls rather than lowlevel Clibrary file descriptor stuff. (abs-shell extracts the DOS handles later on) One critical bit is the line "/PIPE/%x%lu/32768/0" translating that back into amiga "PIPE:<name>/<buffersize>/<numbuffers>" Specifying 0 for numbuffers makes the pipe dynamic, it will never fill (subject to available memory) and so you wont get hanging programs. Make buffersize larger if you have more larger amounts of data to pass through data.
Top
Fri, 2015-04-17 22:16
Massi
Massi's picture
Offline
Last seen: 2 years 10 months ago
Joined: 2012-03-28 17:16
@trixie I can' t remember of
@trixie I can' t remember of any OS3 / OS4 specific source code examples, perhaps http://wiki.amigaos.net/wiki/Main_Page can give some hints. Anyway "pipes" are a very common form of IPC (Inter Process Communication) from the Unix world. So it may be of help reading a Linux tutorial or such. The same concepts should apply to OS4.
Top
Sun, 2015-04-19 11:44
Massi
Massi's picture
Offline
Last seen: 2 years 10 months ago
Joined: 2012-03-28 17:16
@trixie Some more bits from
@trixie Some more bits from my Linux experience, the same concepts should apply to OS4. The synchronism between the communicating processes (2 or more) is based on the following: 1) a process gets blocked on trying to write data to a full pipe 2) a process gets blocked on trying to read data from an empty pipe Pipes can only handle a limited amount of data (say 4096 bytes, see #define PIPE_BUF in limits.h), however the system guarantees the whole data transfer or nothing. Pipes are very much suitable for applications based on Boss-Worker / Producer-Consumer model, where processes can easily added / removed at run time.
Top
Mon, 2015-04-20 10:37
(Reply to #4)
trixie
trixie's picture
Offline
Last seen: 2 days 15 hours ago
Joined: 2011-02-03 13:58
@broadblues Ugh how unix
@broadblues
Ugh how unix :-)
It is, isn't it? :-D
FUNCX;
What does this actually do, or stand for? @Massi Thanks for the tips!
AmigaOne X5000-020 / 2GB RAM / Sapphire Pulse Radeon RX 560 / AmigaOS 4.1 Final Edition Update 2
Top
Mon, 2015-04-20 20:37
(Reply to #5)
broadblues
broadblues's picture
Offline
Last seen: 3 years 1 month ago
Joined: 2012-05-02 21:48
@trixie just debug macro
@trixie just debug macro ignore it :-)
Top
Thu, 2015-04-23 00:31
(Reply to #6)
xenic
xenic's picture
Offline
Last seen: 11 months 23 hours ago
Joined: 2011-05-07 04:52
@trixie Check out the
@trixie Check out the queue.handler documentation in SYS:Documentation/handlers. The big problem might be getting lame etc. to work with an Amiga pipe.
X1000 - OS 4.1FE
Top
Thu, 2015-04-23 12:45
(Reply to #7)
broadblues
broadblues's picture
Offline
Last seen: 3 years 1 month ago
Joined: 2012-05-02 21:48
The big problem might be
The big problem might be getting lame etc. to work with an Amiga pipe.
No problem there. Amiga pipes are quite functional, and so is lame. 
  1. 9.AmigaOS4:> type Music:AudioEvolution4/Projects/08_04_30_Jazz_Tune/Jazztune_08_04_30.aiff | Music:Lame/lame  - >ram:out.mp3
  2. LAME 3.98.2 32bits (http://www.mp3dev.org/)
  3. Using polyphase lowpass filter, transition band: 16537 Hz - 17071 Hz
  4. Encoding <stdin> to <stdout>
  5. Encoding as 44.1 kHz j-stereo MPEG-1 Layer III (11x) 128 kbps qval=3
I can verify that ram:out.mp3 contained a viable mp3.
Top
Thu, 2015-04-23 12:54
(Reply to #8)
broadblues
broadblues's picture
Offline
Last seen: 3 years 1 month ago
Joined: 2012-05-02 21:48
And for prgrams that don't
And for prgrams that don't support stdon/stdout but need a file name 
  1. 9.AmigaOS4:> run >NIL: type Music:AudioEvolution4/Projects/08_04_30_Jazz_Tune/Jazztune_08_04_30.aiff >PIPE:foo/32768/0 
  2. 9.AmigaOS4:> Music:Lame/lame  PIPE:foo ram:out.mp3
  3. LAME 3.98.2 32bits (http://www.mp3dev.org/)
  4. Using polyphase lowpass filter, transition band: 16537 Hz - 17071 Hz
  5. Encoding PIPE:foo to ram:out.mp3
  6. Encoding as 44.1 kHz j-stereo MPEG-1 Layer III (11x) 128 kbps qval=3
  7.     Frame          |  CPU time/estim | REAL time/estim | play/CPU |    ETA 
  8.   7995/7995  (100%)|    1:11/    1:11|    1:11/    1:11|   2.9341x|    0:00 
  9. -------------------------------------------------------------------------------
  10.    kbps        LR    MS  %     long switch short %                             
  11.   128.0        2.8  97.2        97.2   1.7   1.1                               
  12. Writing LAME Tag...done
  13. ReplayGain: -4.0dB
  14. 9.AmigaOS4:>
Obviously you would add *>NIL: to deal with stderr output, or find the "quiet" switdh" I just left that out so show the commands were working.
Top
Thu, 2015-04-23 16:44
(Reply to #9)
trixie
trixie's picture
Offline
Last seen: 2 days 15 hours ago
Joined: 2011-02-03 13:58
@broadblues Thanks! Now I
@broadblues Thanks! Now I only need to work out how to pipe the individual data portions so that lame receives a continuous stream representing the entire audio track.
AmigaOne X5000-020 / 2GB RAM / Sapphire Pulse Radeon RX 560 / AmigaOS 4.1 Final Edition Update 2
Top
Sun, 2015-05-03 15:03
(Reply to #10)
ChrisH
ChrisH's picture
Offline
Last seen: 2 years 7 months ago
Joined: 2010-12-07 20:22
I'm currently using a named
I'm currently using a named pipe (along with Curl) to handle httpS network connections. I'd be very interested to know if it's possible to have ANONYMOUS pipes on the Amiga. i.e. Connect two programs together, without having to give a specific "file" name to Pipe:. Currently I generate a "random" 8-digit hexadecimal name, and hope that it's unique.
Author of the PortablE programming language. I love using Amiga OS4.1 on my X1000 & Sam440 :-D
Top
Sun, 2015-05-03 21:02
(Reply to #11)
broadblues
broadblues's picture
Offline
Last seen: 3 years 1 month ago
Joined: 2012-05-02 21:48
Ofcourse you can open an
Ofcourse you can open an anonymous pipe: as this short shell session indicates: New Shell process 9 9.AmigaOS4:> echo foo >PIPE: 9.AmigaOS4:> type PIPE: foo 9.AmigaOS4:> But anonymous pipes like that are only suitable for use in shell sessions where you can be reasonably guarenteed that no other program is going to try and open an anonymous pipe in te gap between you creating the write end and the read end. To be sure you are opening your own pipe you *want* to use a named pipe. But if you don't want to 'think' of your own name version 50 offers the UNIQUE switch.
/UNIQUE - Ignore the channel name and pick a unique name for this pipe instead. This is useful for applications which pipe input from one program to another. You create the pipe with the /UNIQUE parameter, query the pipe's name using the ExamineObject() function and then open the other pipe using the name obtained (V50).
Top
Mon, 2015-05-04 11:22
(Reply to #12)
ChrisH
ChrisH's picture
Offline
Last seen: 2 years 7 months ago
Joined: 2010-12-07 20:22
But anonymous pipes like that
But anonymous pipes like that are only suitable for use in shell sessions where you can be reasonably guarenteed that no other program is going to try and open an anonymous pipe in te gap between you creating the write end and the read end.
OK, thanks. I wasn't sure what would happen if I (successfully) started reading + writing anonymously using a pipe, and then in the middle of using it another program started using another anonymous pipe. From what you say, it will get an entirely separate/new anonymous pipe :-) . BTW, it appears not possible to see what (named) pipes already exist? Otherwise I could easily generate my own non-colliding name...
Author of the PortablE programming language. I love using Amiga OS4.1 on my X1000 & Sam440 :-D
Top
Log in or register to post comments