FreeBSDwiki - User contributions [en]

Gvinum

2007-08-15T21:31:11Z

Ninereasons: an LVM

(NOTE: for practical tasks, you should probably see [[gmirror]] or [[graid3]] instead of this article.)

'''{{PAGENAME}}''' is a Logical Volume Manager, providing kernel-level software [[RAID]] for the FreeBSD (and NetBSD) operating systems. ''Gvinum'' is the replacement, since FreeBSD 5.3, for the [http://www.vinumvm.org/ ''Vinum'' Volume Manager], re-written to utilize [[geom]] communication methods, and is included in the FreeBSD core distribution. Many of the ''vinum'' functions are reproduced in ''gvinum'', but not all.

==External links==
* The fourth chapter, concerning Vinum, from [http://www.oreilly.com/catalog/cfreebsd/index.html The Complete FreeBSD] is available on-line as [http://www.vinumvm.org/cfbsd/vinum.pdf PDF] or [http://www.vinumvm.org/cfbsd/vinum.txt text]

[[Category:FreeBSD for Servers]]
[[Category:System Commands]]
[[Category:RAID]]

User talk:Ninereasons

2007-08-15T21:26:57Z

Ninereasons: /* btw, thought you'd be interested to know */

== Sure, I'll take it... ==

Your documentation is welcome, as long as you provide some background info on what you're documenting... perhaps I should blush, but I'm not even sure what an LSI device is, exactly. =) Glad you've found the wiki useful! --[[User:Jimbo|Jimbo]] 08:11, 6 April 2006 (EDT)

: Goodness, no reason to blush. I hope this material isn't too obscure. It seemed to me that since your site has search engine presence, the documentation might be found and improved more readily if it were parked here. Thank you for accomodating me - for the good of the community and all that! [[User:Ninereasons|Ninereasons]] 14:26, 6 April 2006 (EDT)

: I wanted to assure you in passing that, I have no connection whatever with LSI Logic or megarc, other than as a user who found FreeBSD support of these cards a bit scanty. I was frustrated by the fact that it took me a good part of two days to find out what the alarm codes meant, and how to bring a replacement drive online without rebooting. In fact, I thought at first that the alarm was coming from the motherboard, because the drive would not rebuild, and the machine would not boot while the volume was degraded. I was within a hair's breadth of trying to replace the board! So, I'm the one who should blush. [[User:Ninereasons|Ninereasons]] 13:33, 7 April 2006 (EDT)

Wouldn't bother me a bit if you WERE connected to LSI Logic or megarc, as long as your documentation was useful, pertinent, and not full of "omg you gots to have teh megarc it is teh K-RAD!!!1" :) --[[User:Jimbo|Jimbo]] 13:44, 7 April 2006 (EDT)

== so....what is an LSI? ==

i don't deal with storage, so i've no idea...

--[[User:Dave|Dave]] 22:51, 6 April 2006 (EDT)

: "LSI Logic" is a manufacturer/marketer of consumer product controller chips and storage devices. [[User:Ninereasons|Ninereasons]] 13:10, 7 April 2006 (EDT)

== machine names and style consistency ==

As long as the machine name stays consistent within a single article, there is no one single machine name we particularly want in every single article... at least, not at this time. If we were going to standardize every single article to a single machine name, we'd probably want to pick something fictitious anyway.

If you come across an article that refers to ph34r in one block and samizdata or oyabun (or whatever) later on - and it's ''not'' supposed to be an article about two different machines in the first place - please jump right on in and sanitize it. But otherwise leave 'em like it is please. =) --[[User:Jimbo|Jimbo]] 20:36, 16 May 2006 (EDT)

: Alrighty. [[User:Ninereasons|Ninereasons]] 11:17, 17 May 2006 (EDT)

== btw, thought you'd be interested to know ==

I'm getting about 20 or so search engine hits per month on "megarc", so it would seem your articles are serving the community well. =) --[[User:Jimbo|Jimbo]] 10:25, 25 May 2007 (EDT)
: I'm glad to hear that, Jimbo. Thanks for letting me know. [[User:Ninereasons|Ninereasons]] 17:26, 15 August 2007 (EDT)

Tomcat

2006-10-21T20:44:08Z

Ninereasons: /* Setup Environment Variables */

Tomcat is used as a server to run Java Servlets. It can be used stand alone or through Apache. Java Servlets is a great language which is very flexible. You can perform balance loading and connect to many great database engines such as postgres. There are many frameworks which you can use to speed up development and keep your code organised, a popular framework is 'struts'.

In this document I will explain how to setup jakarta-tomcat-5.5

==Installation==
===Versions===
The packages and version numbers I will be installing/working with here are:
<li>[http://www.freebsdwiki.net/index.php/Installing_FreeBSD_-_Standard_Installation FreeBSD 6.1]
<li>jakarta-tomcat-5.5.17_1
<li>jakarta-struts 1.2.4
<li>apache-ant-1.6.5_1
<li>diablo-jdk-freebsd6.i386.1.5.0.07.00
<li>[http://www.freebsdwiki.net/index.php/PostgreSQL PostgreSQL 8.1.4]
<li>KATE 2.5.3

===Install Java===
cd /usr/ports/java/diablo-jdk15/ && make install clean
Java will require you to download a heap of files separately through the browser and place them in /usr/ports/distfiles

===Install Tomcat===
Install tomcat from ports as the post-install scripts help configure the system faster.
# cd /usr/ports/*/tomcat55
# make install

===Install ANT===
Used for compiling
# pkg_add -r apache-ant

==Configure==
There is a lot of configuration and tests which you can do in tomcat. Here I'm going to show a basic setup in which you should be able to start some projects.

===Startup===
If you want all the servers to load up on boot here is what you have to add to /etc/rc.conf
tomcat55_enable="YES"
postgresql_enable="YES"

===Setup Environment Variables===
Editing /etc/profile
JAVA_HOME=/usr/local/diablo-jdk1.5.0; export JAVA_HOME
CATALINA_HOME="/usr/local/tomcat5.5"; export CATALINA_HOME
CATALINA_BASE="/usr/local/tomcat5.5"; export CATALINA_BASE
CATALINA_TMPDIR="/usr/local/tomcat5.5/temp"; export CATALINA_TMPDIR

CLASSPATH="/sbin:/bin:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin:/usr/local/tomcat5.5/common/lib/servlet-api.jar:
/usr/local/tomcat5.5/common/lib/jsp-api.jar:.";export CLASSPATH

===Tomcat Servlet Reloading===
So that your modifications are automatically reloaded on your development box edit:
/usr/local/tomcat5.5/conf/context.xml and change:
<Context>
to
<Context reloadable="true">

===First time startup===
To start the tomcat server for the first time here is what you do:
shell# cd /usr/local/tomcat5.5/bin
shell# bash
shell# JAVA_HOME=”/usr/local/diablo-jdk1.5.0”;export JAVA_HOME
shell# ./startup.sh

==Using Tomcat==
The default for tomcat is http://<ip>:8180 (used to be on port 8080)

The work area is /usr/local/tomcat5.5/webapps

You can enter project_name.war files into the webapps directory then access them like so:
http://<ip>:8180/project_name

[[Category : Ports and Packages]]

Megarc

2006-07-24T17:41:11Z

Ninereasons: /* Documentation */

[[Category:Ports and Packages]] [[Category:RAID]]
'''''{{PAGENAME}}''''' is a commandline utility that provides an interface to many of the configuration and reporting functions for LSI Logic's MegaRAID BIOS (http://www.lsilogic.com).

==Megarc port for FreeBSD==
The ''megarc'' utility ships as a binary, without any accompanying documentation, on the "Megaraid Universal Software Suite" CD which accompanies any boxed LSI Logic RAID storage adapter. It is also found in the FreeBSD ports collection.

* On the CD that comes with the storage adapter, the utility is found in two places. It is part of a zipped bundle in <cd>:\SW_Components\Drivers\dr_freebsd_1.51.zip on our distribution. The zip file contents are as follows:

<pre>
Length Date Time Name
-------- ---- ---- ----
130448 04-18-05 14:10 MegaRC 1.04.zip
12953 10-08-04 16:05 amr_x86_64_ver1_51FreeBSD5.3.tgz
153600 03-17-05 15:19 code.tar
535 02-10-05 11:02 FreeBSDDriverUpdate.txt
12774 10-12-04 19:38 amr_i386_ver_1_51_FreeBSD5.3.tgz
-------- -------
310310 5 files
</pre>

* The utility is also found on the CD by itself, at "<cd>:\SW_Components\Utilities\ut_FreeBSD_MegaRC 1.04.zip".

* It can be also be installed from the [http://www.freebsd.org/cgi/cvsweb.cgi/ports/sysutils/megarc/ ports collection] at /usr/ports/sysutils/megarc. The port extracts the binary from the bundled dr_freebsd_*.zip Our example was installed from the ports collection on 6.1-PRERELEASE FreeBSD amd64

<pre>
# file /usr/local/sbin/megarc
/usr/local/sbin/megarc: ELF 32-bit LSB executable, Intel 80386,
version 1 (FreeBSD), for FreeBSD 5.2.1, statically linked, stripped
</pre>

==Documentation==
The documentation for megarc is limited to the output of its ?/help functions. Since these [[RAID]] adapters are reasonably popular, and the commands are a bit scary and somewhat obscurely named, this page might prove helpful to someone even though it's not complete.

Megarc must be run with sufficient privileges, or the adapter will not be found. "?" and "help" are not synonymous. ''?'' (mis-named "complete help") gives brief syntax hints. ''help'' gives a fuller description of what the command does, and its arguments and usage.

* To see the commands available, as root type: <code>megarc ?</code>
First Parameter should be :
-AllAdpInfo
... etc ...
*To see a list of commands and syntax, type: <code>megarc help</code>
<pre>
-------------------------------------------------------------------------
* Convention Used:
[Cmd = The name of the utility]
opt1|opt2|opt3 => Only one of these can be specified

-------------------------------------------------------------------------
Cmd ?: Gives complete help
Cmd -functionality ?: functionality specific help
Example:
Cmd -ctlrInfo ? :help on -ctlrinfo option

-------------------------------------------------------------------------
Usage: Cmd -ctlrInfo -aX
... etc ...
</pre>
* '''Example:''' <code> megarc -ctlrInfo help </code>
<pre>
**********************************************************************

usage :
cmd -ctlrInfo -aX
: Shows general adapter info of adapter
**********************************************************************
where :
cmd : name of the utility
-aX : adapter number X(max 12 adapters). X=0..11
**********************************************************************
</pre>

==Examples==
For the examples below, we are using an LSILogic 150-6 x64 SATA RAID adapter on a 32-bit PCI slot - "adapter 0" on the system - with a RAID 5 volume consisting of five (5) physical SATA drives of 400GB each. The hot-spare belongs at target 00 but is being replaced at the time of this writing.

Below are some brief descriptions of some of the commands, and some examples of output from a few of the information functions. These command-options are not case-sensitive.

===megarc -dispCfg -a0 ===
Display the configuration for adapter ''0''.

<pre>
Logical Drive : 0( Adapter: 0 ): Status: OPTIMAL
---------------------------------------------------
SpanDepth :01 RaidLevel: 5 RdAhead : No Cache: DirectIo
StripSz :064KB Stripes : 5 WrPolicy: WriteThru

Logical Drive 0 : SpanLevel_0 Disks
Chnl Target StartBlock Blocks Physical Target Status
---- ------ ---------- ------ ----------------------
0 01 0x00000000 0x2e936800 ONLINE
0 02 0x00000000 0x2e936800 ONLINE
0 03 0x00000000 0x2e936800 ONLINE
0 04 0x00000000 0x2e936800 ONLINE
0 05 0x00000000 0x2e936800 ONLINE
</pre>

===megarc -LogPhysInfo -a0 ===
Display the physical drive information for each of the logical drives on adapter ''0''.

<pre>
Logical drive 0: RaidLevel 5

Physical Drive Information
Channel 0
381549MB drive ID 1
CoerSZ: 781412352(Sectors) 381549(MB) RawSZ: 781422255(Sectors)
381549MB drive ID 2
CoerSZ: 781412352(Sectors) 381549(MB) RawSZ: 781422255(Sectors)
381549MB drive ID 3
CoerSZ: 781412352(Sectors) 381549(MB) RawSZ: 781422255(Sectors)
381549MB drive ID 4
CoerSZ: 781412352(Sectors) 381549(MB) RawSZ: 781422255(Sectors)
381549MB drive ID 5
CoerSZ: 781412352(Sectors) 381549(MB) RawSZ: 781422255(Sectors)

</pre>

===megarc -ScfgAndParm|-DfcfgAndParm|-RcfgAndParm -fFileName -a0 ===
Save, Display, or Restore the configuration and parameters for adapter ''0'', in ''FileName''. ''FileName'' stores the same output provided by -dispCfg in a binary format, making it possible to directly load the stored configuration from the file.

===megarc -physOn pd[c0:t0,c1:t1....] -a0===

Set the state of the designated drive(s) to ''Online''. ''pd[c:t]'' refers to at least one physical drive by channel and target. ''-aN'' here as elsewere is the adapter number [required]

If the physical drive does not exist or if it isn't in failed state, the utility exits with no harm done.

An example of this command under our present configuration would be:
megarc -physOn -a0 pd[0:1]

===megarc -phys -chAll -idAll -a0===
Show the physical drive description for each device on all channels managed by adapter ''0''

<pre>

Adapter 0, Channel 0, Target ID 1
Type: DISK Vendor : WDC
Product: WD4000KD-00NAB0 Revision : 01.0
Synchronous : No Wide-32 : No Wide-16: No
LinkCmdSupport: No TagQ support: No RelAddr: No
Removable : No SoftReset : No AENC : No
etc...
</pre>

===megarc -physdrvSerialInfo -chAll -idAll -a0===
Show the serial number for each physical drive on each channel for all serial devices managed by adapter ''0'' (This doesn't look correct or helpful).

<pre>
Adapter 0, Channel 0, Target ID 1

PhysDrvSerial#: WD-W

etc ...
</pre>

===megarc -pdFailInfo -chAll -idAll -a0===
Show the failure history for each device on all channels managed by adapter ''0''.

===megarc -setRbldRate|-getRbldRate -a0 ===
Get the rebuild rate for adapter ''0''.
<pre>

# megarc -getRbldRate -a0

...

**********************************************************************
RebuildRate of Adapter-0 is 30
**********************************************************************
</pre>

===megarc -ctlrInfo -a0===
Display information about adapter ''0''.

<pre>
**********************************************************************
Information of Adapter-0 (#Adapter(s) on system: 1)
**********************************************************************

Firmware Version : 713N BIOS Version : G119
Logical Drives : 01 DRAM : 64MB
Rebuild Rate : 30%
Flush Interval : 4 secs
Number Of Chnls : 1 Bios Status : Enabled
Alarm State : Enabled Auto Rebuild : Enabled
FW : SPAN-8, 40-LD BIOS Config AutoSelection : USER
BIOS Echos Mesg : ON BIOS Stops On Error : ON
Initiator Id : 16(Clustered Firmware)
Board SN: -17179869
**********************************************************************
</pre>

===megarc -getXFerRate|-setXFerRate -a0 -chAll===
Get or set the transfer rate for all channels on adapter ''0''.
<pre>
# megarc -getXFerRate -a0 -ch0

**********************************************************************
Transfer Rate of Adapter-0 Channel-0 is 160M
**********************************************************************
</pre>

Redirection

2006-07-12T21:45:56Z

Ninereasons: /* Shell pipelines and redirection */

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Shell redirection operators include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipelines and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). When the next process inherits the open ''fd'' from the shell the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (0-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

=== ''tcsh'' and ''sh'' ===
# Write output to a file
#: % mycmd > out.txt
#:: stderr stuff
# Append output to a file
#: % mycmd >> out.txt
#:: stderr stuff
# Redirect the output of a remote command to ''local.txt''.
#: % localcmd "remotecmd" > local.txt
#:: stderr stuff
# Same command as above, showing only the changes compared to ''local.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin'. These two commands are equivalent. ''
#: % localcmd "remotecmd" | diff /dev/stdin local.txt
#: % localcmd "remotecmd" | diff - local.txt
#:: stderr stuff
# Direct <code>stdout</code>+<code>stderr</code> to file
#: % mycmd >& out.txt
# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
#: % <jumble.txt sort >sorted.txt
#: % >sorted.txt sort <jumble.txt
#: % sort < jumble.txt >sorted.txt
#: % <jumble.txt>sorted.txt sort
#:: stderr stuff
# Sort unique lines of jumble.txt into sorted.txt
#: % <jumble.txt sort | uniq >sorted.txt
#:: stderr stuff
# Sort ''HERE doc'' delimited by "lines"
#: % <<lines sort 
#::? a second line 
#::? a first line 
#::lines a first line a second line stderr stuff

=== ''tcsh'' only ===
# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
#: % ( mycmd > /dev/stdin ) > & /dev/null
#:: stdout stuff
#: % ( mycmd > /dev/tty ) > & /dev/null
#:: stdout stuff
# Append output to out.txt; discard messages
#: % (mycmd >> out.txt) >& /dev/null
# Write output to out.txt; store and watch errors ''Note: Compare the same task in ''sh''. ''
#: % ( mycmd > out.txt ) | & tee err.txt
#:: stderr stuff

=== ''sh'' only ===
# Discard errors, watch output
#: $ mycmd 2> /dev/null
#:: stdout stuff
# Append output to out.txt; discard messages
#: $ mycmd 2> /dev/null >> out.txt
# Write output to out.txt; store and watch errors ''Note: Compare the same task in ''tcsh''. ''
#: $ mycmd 2>&1 > out.txt | tee err.txt
#:: stderr stuff
# Write messages to err.txt; write output to out.txt and copy output to terminal
#: $ mycmd 2> err.txt | tee out.txt
#:: stdout stuff
# Assign a variable from stored.txt
#: $ <stored.txt read var; mycmd $var
#:: stderr stuff
#:: stdout stuff
# Assign first three lines of stored.txt to three different variables
#: $ exec 3<&0; exec <stored.txt; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
#:: stderr stuff
#:: stdout stuff
# Use all unique lines in stored.txt as an argument variable for ''mycmd'', appending to result.txt
#: $ exec 3<&0; exec <stored.txt; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
#:: stderr stuff
: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
:: $ mycmd 1> out.txt
:: $ mycmd > out.txt
::: stderr stuff
: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
:: $ mycmd 0< in.txt
:: $ mycmd < in.txt
::: stderr stuff
::: stdout stuff
: ''Note:'' To close a file descriptor, say <code>n<&-</code>
:: $ mycmd >out.txt 1<&-;
:: ''-sh: fails without messages''
:: -bash: mycmd: write error: Bad file descriptor

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to another command. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 | command > errors.txt
because when you switched output to the messages file descriptor, it's done ''right away''. When you tried to switch errors to output there's nothing left to pipe. It's exactly as though you had said
samizdata# myprogram 1>/dev/stderr | command > errors.txt
''command'' receives no data, because it's all been sent to your screen. ''errors.txt'' will be empty. If you had tried it the other way, it's just as though you would have said:
samizdata# myprogram 2>/dev/stdout | command > errors.txt
In which case, errors and output are both piped to ''command'', which was not what you wanted.

This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, like this:
samizdata# myprogram 3>&2 2>&1 1>&3 | command > errors.txt

Let's step through what this does.
# ''fd 3'' points to <code>stderr</code>
# ''fd 2'' points to <code>stdout</code>
# ''fd 1'' points to ''fd 3'' (which is connected to <code>stderr</code>)
# ''|'' passes the open file descriptor connected to <code>stdout</code> (''fd 2'') making it <code>stdin</code> for ''command''
# ''>'' redirects the output of ''command'' to ''errors.txt''.

So, what you'll see on your screen is the output of ''myprogram'' on <code>stderr</code> (not passed to ''command''), followed by errors (if any) from ''command''. In short, what's been accomplished is, a swap of errors for output, from ''myprogram'' to ''command'', the output of which is redirected to ''errors.txt''. There is no reliable way to do the same thing in <code>tcsh</code>.

Also, see the [http://www.freebsd.org/cgi/man.cgi?query=mkfifo&apropos=0&sektion=1&manpath=FreeBSD+6.1-RELEASE&format=html man page] for [[mkfifo]], a utility for creating arbitrary file descriptor files.
[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Talk:Portupgrade

2006-07-05T05:04:26Z

Ninereasons: /* Putting the pieces together for portupgrade */

note on the rollback: I strongly prefer showing a pipe to grep rather than the -l argument because they both accomplish the same thing, but piping to grep is a universal tool than can be used with any command or process. It's a good habit to learn using tools and skills that are universally applicable as opposed to stuff that's only applicable to one particular task, unless there is some marked advantage to using the more proprietary method of doing things.

If you know of some particularly good reason to use the -l argument instead of piping to grep, by all means let me know here. --[[User:Jimbo|Jimbo]] 19:23, 9 Jul 2005 (EDT)

----
----

The goal is just avoiding to use two processes when only one is needed... More, I tried this :
Script started on Sun Jul 10 11:35:19 2005
# time portversion -l '<' > /dev/null
portversion 1.90s user 0.68s system 86% cpu 3.001 total
# time portversion | grep '<' > /dev/null
portversion 2.68s user 0.88s system 69% cpu 5.102 total
grep '<' > /dev/null 0.00s user 0.00s system 0% cpu 5.100 total
Script done on Sun Jul 10 11:36:23 2005

I do not know how it work internaly, but I gess that the "-l" argument changes the search algorithme.

But I agree with your point of view... grep has to be known by the users !
The "-l" flag may sometimes be preferable, for a low system for example...

----
----

Yah, if I was scripting repeated operations, by all means I'd use the more efficient way. But in general I think the modularity of *nix is actually by far the best thing about it, and I try to do things as modular as possible if there isn't a compelling reason to do otherwise. Even in my own just day-to-day stuff; sure I already know about grep, but still actually using it for everything keeps me in the habit of using it and that's more useful to me than being in the habit of using something that only applies to portversion, y'know? =)

Btw, the difference in efficiency isn't anywhere near as compelling on my systems:

ph34r# /usr/bin/time -h portversion -l '<' > /dev/null
0.88s real 0.70s user 0.18s sys
ph34r# /usr/bin/time -h portversion | grep '<' > /dev/null
0.89s real 0.68s user 0.21s sys

or

blackbox# /usr/bin/time -h portversion -l '<' > /dev/null
1.03s real 0.83s user 0.18s sys
blackbox# /usr/bin/time -h portversion | grep '<' > /dev/null
1.04s real 0.83s user 0.19s sys

On looking again at your times, I notice that the CPU% is quite a bit higher on the faster score. I have a sneaking suspicion a cron job or something kicked in while you were running the second one, so it just didn't get as much processor time and took longer.

--[[User:Jimbo|Jimbo]] 11:52, 10 Jul 2005 (EDT)

----
----

As my load average is about 1.0, I have done my tests 4 or 5 time to check that the result was always the same (it was)... But it may not have incidence on the statistics since they are updating by the kernel when the process is effectively running and collected by the '''time''' command by reading a data structure filled by the ''wait4()'' system call (If I don't remember wrong, '''time''' ''fork()'' itself, ''exec()'' the command and ''wait4()'' for it)...

What I think is that the execution time can be dependant of the number of results : As I might have everyting up to date, i might have the greater difference I can get (all the ports vs. none)...

Well... I have nothing to conclude from this ... All the two methods do the same :)
--[[User:Smortex|Smortex]]

----
----

I'm betting it has to do with your high average load. I generally work with fairly high-powered servers in relatively small environments, with load frequently averaging as low as .05 even during working times. (PS: when you're writing in talk pages, you might want to use the signature button (looks like a little squiggle up at the top of the textarea) to sign your comments, or if you're using a non-graphical browser, you can get the same effect by using <nowiki>--~~~~</nowiki> as a code.) =) --[[User:Jimbo|Jimbo]] 18:55, 10 Jul 2005 (EDT)

----
----
05/30/06
I notice that there is no mention to doing a CVS before using portupgrade; also no mention of the INDEX file being remove. Should these aspects be addressed?

----
----
absolutely... address 'em! --[[User:Jimbo|Jimbo]] 00:14, 31 May 2006 (EDT)

----
----

I just started to use FreeBSD again; so I have to do some research about it before I make a comment. Thanks Jimbo. Wed, June 7, 2006 7:51 AM CDT.

----
----

==Putting the pieces together for portupgrade==

Portupgrade depends on INDEX-6 and INDEX-6.db to function properly. As of release 5.4; INDEX-6 and INDEX-6.db are not part of the download from CVSUP. You have to build or make the INDEX* files.

1) cvsup -g -L 2 ports-supfile

2) Update the INDEX files via the three methods listed below (a,b,c):
a) cd /usr/ports, then "make index"
b) cd /usr/ports, then "makefetch index"
c) cd /usr/ports, then portsdb -Uu
3) Then upgrade your ports as described in the methods described above.

A caveat should be mentioned about the methods of obtaining the INDEX files. Option a) "make index" is officially supported by FreeBSD. Option b) is supported by FreeBSD; however, please keep in mind that the INDEX files that are retrieved from the servers may be slightly out of date; which may cause build problems. You may have to change the premission on the download INDEX files via (chmod to match the premissions: "-rw-r--r--". Option c) portsdb -Uu was never offically announced as being acceptable or not acceptable. Portsdb -Uu does create/build and update the INDEX.

My method of upgrading ports is:
Step 1, step 2c and then step 3.

The approved methods can be found here as well as the blurb about removing the INDEX files from CVSUP in section 2.5:

http://www.freebsd.org/releases/5.4R/relnotes-i386.html

--[[User:68.164.11.90|68.164.11.90]] 15:54, 2 July 2006 (EDT)Hope this helps.
Pete

: Thanks for the notes. It would be helpful to add them in to the article. The target is ''make fetchindex'' (rather than ''makefetch index''). Running ''make index'' builds an up-to-the-minute Index. You might also mention that, ''make index'' is very, very slow. <code>portsdb -U</code> "creates/updates the Index" by a system call to <code>make index</code>, so it's just as slow. There's a port in sysutils/p5-FreeBSD-Portindex, which is a set of perl scripts that incrementally keeps the Index updated, by using a cache of previous output. At least in theory, this should be faster; but I haven't used the port.

:Portupgrade runs <code>portsdb -u</code> as part of its routine before building, and between each build - so, I can't see the advantage of the extra precaution of running ''portsdb -U'''u''''' before running portupgrade (''portsdb -U'' would seem sufficient). On the other hand, this is the procedure recommended on the [http://www.freebsd.org/cgi/man.cgi?query=portsdb&apropos=0&sektion=0&manpath=FreeBSD+6.1-RELEASE+and+Ports&format=html portsdb man page], and the ''-u'' doesn't cost a lot of time. [[User:Ninereasons|Ninereasons]] 13:53, 4 July 2006 (EDT)

:: Hello Ninereasons, I put this in the discussion section; to see if there could be a consensus of users; aka a peer review of bsd users. I am somewhat hesitant to make any official changes to a wiki until I get some feedback as well as proof reading what I have written (IE: the "make fetchindex" typing error). As for the extra tools; with respect to managing / updating the index; I would like to see this as bare metal as possible. In other words; the tools exist to update the index; no need to add more tools (imho). I am trying to look at it from a new users perspective. I could very well be wrong or misguided in that view. PS: never used a wiki before; I don't wish to break the original documentation. PSS: thanks for the feedback.
--[[User:68.164.11.90|68.164.11.90]] 21:49, 4 July 2006 (EDT)Pete

::: Hi Pete. You can call me "Mark". The beauty of a wiki is that you won't "break the original" by making a useful contribution like the one you're contemplating. If you make mistakes, we'll fix them. If you seem to be wrong, which isn't the case here, we would just restore what you messed up - and no hard feelings. I'm a guest here, just like you. But even as a guest, I can invite you to log in with a username and give it a try. [[User:Ninereasons|Ninereasons]] 01:03, 5 July 2006 (EDT)

== hey Pete ==

How about registering an account and using it, if you plan on being a recurring contributor?

It's not ''required'' by any means, but it makes it a lot easier for the rest of us to keep track of who's who (and whether or not a new edit is likely to be spam or vandalism).

Thanks, and welcome to the wiki! --[[User:Jimbo|Jimbo]] 23:01, 4 July 2006 (EDT)

Talk:Portupgrade

2006-07-05T05:03:35Z

Ninereasons: /* Putting the pieces together for portupgrade */

note on the rollback: I strongly prefer showing a pipe to grep rather than the -l argument because they both accomplish the same thing, but piping to grep is a universal tool than can be used with any command or process. It's a good habit to learn using tools and skills that are universally applicable as opposed to stuff that's only applicable to one particular task, unless there is some marked advantage to using the more proprietary method of doing things.

If you know of some particularly good reason to use the -l argument instead of piping to grep, by all means let me know here. --[[User:Jimbo|Jimbo]] 19:23, 9 Jul 2005 (EDT)

----
----

The goal is just avoiding to use two processes when only one is needed... More, I tried this :
Script started on Sun Jul 10 11:35:19 2005
# time portversion -l '<' > /dev/null
portversion 1.90s user 0.68s system 86% cpu 3.001 total
# time portversion | grep '<' > /dev/null
portversion 2.68s user 0.88s system 69% cpu 5.102 total
grep '<' > /dev/null 0.00s user 0.00s system 0% cpu 5.100 total
Script done on Sun Jul 10 11:36:23 2005

I do not know how it work internaly, but I gess that the "-l" argument changes the search algorithme.

But I agree with your point of view... grep has to be known by the users !
The "-l" flag may sometimes be preferable, for a low system for example...

----
----

Yah, if I was scripting repeated operations, by all means I'd use the more efficient way. But in general I think the modularity of *nix is actually by far the best thing about it, and I try to do things as modular as possible if there isn't a compelling reason to do otherwise. Even in my own just day-to-day stuff; sure I already know about grep, but still actually using it for everything keeps me in the habit of using it and that's more useful to me than being in the habit of using something that only applies to portversion, y'know? =)

Btw, the difference in efficiency isn't anywhere near as compelling on my systems:

ph34r# /usr/bin/time -h portversion -l '<' > /dev/null
0.88s real 0.70s user 0.18s sys
ph34r# /usr/bin/time -h portversion | grep '<' > /dev/null
0.89s real 0.68s user 0.21s sys

or

blackbox# /usr/bin/time -h portversion -l '<' > /dev/null
1.03s real 0.83s user 0.18s sys
blackbox# /usr/bin/time -h portversion | grep '<' > /dev/null
1.04s real 0.83s user 0.19s sys

On looking again at your times, I notice that the CPU% is quite a bit higher on the faster score. I have a sneaking suspicion a cron job or something kicked in while you were running the second one, so it just didn't get as much processor time and took longer.

--[[User:Jimbo|Jimbo]] 11:52, 10 Jul 2005 (EDT)

----
----

As my load average is about 1.0, I have done my tests 4 or 5 time to check that the result was always the same (it was)... But it may not have incidence on the statistics since they are updating by the kernel when the process is effectively running and collected by the '''time''' command by reading a data structure filled by the ''wait4()'' system call (If I don't remember wrong, '''time''' ''fork()'' itself, ''exec()'' the command and ''wait4()'' for it)...

What I think is that the execution time can be dependant of the number of results : As I might have everyting up to date, i might have the greater difference I can get (all the ports vs. none)...

Well... I have nothing to conclude from this ... All the two methods do the same :)
--[[User:Smortex|Smortex]]

----
----

I'm betting it has to do with your high average load. I generally work with fairly high-powered servers in relatively small environments, with load frequently averaging as low as .05 even during working times. (PS: when you're writing in talk pages, you might want to use the signature button (looks like a little squiggle up at the top of the textarea) to sign your comments, or if you're using a non-graphical browser, you can get the same effect by using <nowiki>--~~~~</nowiki> as a code.) =) --[[User:Jimbo|Jimbo]] 18:55, 10 Jul 2005 (EDT)

----
----
05/30/06
I notice that there is no mention to doing a CVS before using portupgrade; also no mention of the INDEX file being remove. Should these aspects be addressed?

----
----
absolutely... address 'em! --[[User:Jimbo|Jimbo]] 00:14, 31 May 2006 (EDT)

----
----

I just started to use FreeBSD again; so I have to do some research about it before I make a comment. Thanks Jimbo. Wed, June 7, 2006 7:51 AM CDT.

----
----

==Putting the pieces together for portupgrade==

Portupgrade depends on INDEX-6 and INDEX-6.db to function properly. As of release 5.4; INDEX-6 and INDEX-6.db are not part of the download from CVSUP. You have to build or make the INDEX* files.

1) cvsup -g -L 2 ports-supfile

2) Update the INDEX files via the three methods listed below (a,b,c):
a) cd /usr/ports, then "make index"
b) cd /usr/ports, then "makefetch index"
c) cd /usr/ports, then portsdb -Uu
3) Then upgrade your ports as described in the methods described above.

A caveat should be mentioned about the methods of obtaining the INDEX files. Option a) "make index" is officially supported by FreeBSD. Option b) is supported by FreeBSD; however, please keep in mind that the INDEX files that are retrieved from the servers may be slightly out of date; which may cause build problems. You may have to change the premission on the download INDEX files via (chmod to match the premissions: "-rw-r--r--". Option c) portsdb -Uu was never offically announced as being acceptable or not acceptable. Portsdb -Uu does create/build and update the INDEX.

My method of upgrading ports is:
Step 1, step 2c and then step 3.

The approved methods can be found here as well as the blurb about removing the INDEX files from CVSUP in section 2.5:

http://www.freebsd.org/releases/5.4R/relnotes-i386.html

--[[User:68.164.11.90|68.164.11.90]] 15:54, 2 July 2006 (EDT)Hope this helps.
Pete

: Thanks for the notes. It would be helpful to add them in to the article. The target is ''make fetchindex'' (rather than ''makefetch index''). Running ''make index'' builds an up-to-the-minute Index. You might also mention that, ''make index'' is very, very slow. <code>portsdb -U</code> "creates/updates the Index" by a system call to <code>make index</code>, so it's just as slow. There's a port in sysutils/p5-FreeBSD-Portindex, which is a set of perl scripts that incrementally keeps the Index updated, by using a cache of previous output. At least in theory, this should be faster; but I haven't used the port.

:Portupgrade runs <code>portsdb -u</code> as part of its routine before building, and between each build - so, I can't see the advantage of the extra precaution of running ''portsdb -U'''u''''' before running portupgrade (''portsdb -U'' would seem sufficient). On the other hand, this is the procedure recommended on the [http://www.freebsd.org/cgi/man.cgi?query=portsdb&apropos=0&sektion=0&manpath=FreeBSD+6.1-RELEASE+and+Ports&format=html portsdb man page], and the ''-u'' doesn't cost a lot of time. [[User:Ninereasons|Ninereasons]] 13:53, 4 July 2006 (EDT)

:: Hello Ninereasons, I put this in the discussion section; to see if there could be a consensus of users; aka a peer review of bsd users. I am somewhat hesitant to make any official changes to a wiki until I get some feedback as well as proof reading what I have written (IE: the "make fetchindex" typing error). As for the extra tools; with respect to managing / updating the index; I would like to see this as bare metal as possible. In other words; the tools exist to update the index; no need to add more tools (imho). I am trying to look at it from a new users perspective. I could very well be wrong or misguided in that view. PS: never used a wiki before; I don't wish to break the original documentation. PSS: thanks for the feedback.
--[[User:68.164.11.90|68.164.11.90]] 21:49, 4 July 2006 (EDT)Pete

::: Hi Pete. You can call me "Mark". The beauty of a wiki is that you won't "break the original" by making a useful contribution like the one you're contemplating. If you make mistakes, we'll fix them. If you seem to be wrong, which isn't the case here, we would just restore what you messed up - and no hard feeligns. I'm a guest here, just like you. But even as a guest, I can invite you to log in and give it a try. [[User:Ninereasons|Ninereasons]] 01:03, 5 July 2006 (EDT)

== hey Pete ==

How about registering an account and using it, if you plan on being a recurring contributor?

It's not ''required'' by any means, but it makes it a lot easier for the rest of us to keep track of who's who (and whether or not a new edit is likely to be spam or vandalism).

Thanks, and welcome to the wiki! --[[User:Jimbo|Jimbo]] 23:01, 4 July 2006 (EDT)

Talk:Portupgrade

2006-07-04T19:29:24Z

Ninereasons: /* Putting the pieces together for portupgrade */

Talk:Portupgrade

2006-07-04T19:18:58Z

Ninereasons: /* Putting the pieces together for portupgrade */

note on the rollback: I strongly prefer showing a pipe to grep rather than the -l argument because they both accomplish the same thing, but piping to grep is a universal tool than can be used with any command or process. It's a good habit to learn using tools and skills that are universally applicable as opposed to stuff that's only applicable to one particular task, unless there is some marked advantage to using the more proprietary method of doing things.

If you know of some particularly good reason to use the -l argument instead of piping to grep, by all means let me know here. --[[User:Jimbo|Jimbo]] 19:23, 9 Jul 2005 (EDT)

----
----

The goal is just avoiding to use two processes when only one is needed... More, I tried this :
Script started on Sun Jul 10 11:35:19 2005
# time portversion -l '<' > /dev/null
portversion 1.90s user 0.68s system 86% cpu 3.001 total
# time portversion | grep '<' > /dev/null
portversion 2.68s user 0.88s system 69% cpu 5.102 total
grep '<' > /dev/null 0.00s user 0.00s system 0% cpu 5.100 total
Script done on Sun Jul 10 11:36:23 2005

I do not know how it work internaly, but I gess that the "-l" argument changes the search algorithme.

But I agree with your point of view... grep has to be known by the users !
The "-l" flag may sometimes be preferable, for a low system for example...

----
----

Yah, if I was scripting repeated operations, by all means I'd use the more efficient way. But in general I think the modularity of *nix is actually by far the best thing about it, and I try to do things as modular as possible if there isn't a compelling reason to do otherwise. Even in my own just day-to-day stuff; sure I already know about grep, but still actually using it for everything keeps me in the habit of using it and that's more useful to me than being in the habit of using something that only applies to portversion, y'know? =)

Btw, the difference in efficiency isn't anywhere near as compelling on my systems:

ph34r# /usr/bin/time -h portversion -l '<' > /dev/null
0.88s real 0.70s user 0.18s sys
ph34r# /usr/bin/time -h portversion | grep '<' > /dev/null
0.89s real 0.68s user 0.21s sys

or

blackbox# /usr/bin/time -h portversion -l '<' > /dev/null
1.03s real 0.83s user 0.18s sys
blackbox# /usr/bin/time -h portversion | grep '<' > /dev/null
1.04s real 0.83s user 0.19s sys

On looking again at your times, I notice that the CPU% is quite a bit higher on the faster score. I have a sneaking suspicion a cron job or something kicked in while you were running the second one, so it just didn't get as much processor time and took longer.

--[[User:Jimbo|Jimbo]] 11:52, 10 Jul 2005 (EDT)

----
----

As my load average is about 1.0, I have done my tests 4 or 5 time to check that the result was always the same (it was)... But it may not have incidence on the statistics since they are updating by the kernel when the process is effectively running and collected by the '''time''' command by reading a data structure filled by the ''wait4()'' system call (If I don't remember wrong, '''time''' ''fork()'' itself, ''exec()'' the command and ''wait4()'' for it)...

What I think is that the execution time can be dependant of the number of results : As I might have everyting up to date, i might have the greater difference I can get (all the ports vs. none)...

Well... I have nothing to conclude from this ... All the two methods do the same :)
--[[User:Smortex|Smortex]]

----
----

I'm betting it has to do with your high average load. I generally work with fairly high-powered servers in relatively small environments, with load frequently averaging as low as .05 even during working times. (PS: when you're writing in talk pages, you might want to use the signature button (looks like a little squiggle up at the top of the textarea) to sign your comments, or if you're using a non-graphical browser, you can get the same effect by using <nowiki>--~~~~</nowiki> as a code.) =) --[[User:Jimbo|Jimbo]] 18:55, 10 Jul 2005 (EDT)

----
----
05/30/06
I notice that there is no mention to doing a CVS before using portupgrade; also no mention of the INDEX file being remove. Should these aspects be addressed?

----
----
absolutely... address 'em! --[[User:Jimbo|Jimbo]] 00:14, 31 May 2006 (EDT)

----
----

I just started to use FreeBSD again; so I have to do some research about it before I make a comment. Thanks Jimbo. Wed, June 7, 2006 7:51 AM CDT.

----
----

==Putting the pieces together for portupgrade==

Portupgrade depends on INDEX-6 and INDEX-6.db to function properly. As of release 5.4; INDEX-6 and INDEX-6.db are not part of the download from CVSUP. You have to build or make the INDEX* files.

1) cvsup -g -L 2 ports-supfile

2) Update the INDEX files via the three methods listed below (a,b,c):
a) cd /usr/ports, then "make index"
b) cd /usr/ports, then "makefetch index"
c) cd /usr/ports, then portsdb -Uu
3) Then upgrade your ports as described in the methods described above.

A caveat should be mentioned about the methods of obtaining the INDEX files. Option a) "make index" is officially supported by FreeBSD. Option b) is supported by FreeBSD; however, please keep in mind that the INDEX files that are retrieved from the servers may be slightly out of date; which may cause build problems. You may have to change the premission on the download INDEX files via (chmod to match the premissions: "-rw-r--r--". Option c) portsdb -Uu was never offically announced as being acceptable or not acceptable. Portsdb -Uu does create/build and update the INDEX.

My method of upgrading ports is:
Step 1, step 2c and then step 3.

The approved methods can be found here as well as the blurb about removing the INDEX files from CVSUP in section 2.5:

http://www.freebsd.org/releases/5.4R/relnotes-i386.html

--[[User:68.164.11.90|68.164.11.90]] 15:54, 2 July 2006 (EDT)Hope this helps.
Pete

: Thanks for the notes. It would be helpful to add them in to the article. The target is ''make fetchindex'' (rather than ''makefetch index''). Running ''make index'' builds an up-to-the-minute Index. You might also mention that, ''make index'' is very, very slow. <code>portsdb -U</code> "creates/updates the Index" by a system call to <code>make index</code>, so it's just as slow. There's a port in sysutils/p5-FreeBSD-Portindex, which is a set of perl scripts that incrementally keeps the Index updated, by using a cache of previous output. At least in theory, this should be faster; but I haven't used the port.

:Portupgrade runs <code>portsdb -u</code> as part of its routine before building, and between each build - so, I can't see the advantage of the extra precaution of running ''portsdb -Uu'' before running portupgrade. On the other hand, this is the procedure recommended on the [http://www.freebsd.org/cgi/man.cgi?query=portsdb&apropos=0&sektion=0&manpath=FreeBSD+6.1-RELEASE+and+Ports&format=html portsdb man page]. [[User:Ninereasons|Ninereasons]] 13:53, 4 July 2006 (EDT)

Talk:Portupgrade

2006-07-04T19:01:15Z

Ninereasons: /* Putting the pieces together for portupgrade */

note on the rollback: I strongly prefer showing a pipe to grep rather than the -l argument because they both accomplish the same thing, but piping to grep is a universal tool than can be used with any command or process. It's a good habit to learn using tools and skills that are universally applicable as opposed to stuff that's only applicable to one particular task, unless there is some marked advantage to using the more proprietary method of doing things.

If you know of some particularly good reason to use the -l argument instead of piping to grep, by all means let me know here. --[[User:Jimbo|Jimbo]] 19:23, 9 Jul 2005 (EDT)

----
----

The goal is just avoiding to use two processes when only one is needed... More, I tried this :
Script started on Sun Jul 10 11:35:19 2005
# time portversion -l '<' > /dev/null
portversion 1.90s user 0.68s system 86% cpu 3.001 total
# time portversion | grep '<' > /dev/null
portversion 2.68s user 0.88s system 69% cpu 5.102 total
grep '<' > /dev/null 0.00s user 0.00s system 0% cpu 5.100 total
Script done on Sun Jul 10 11:36:23 2005

I do not know how it work internaly, but I gess that the "-l" argument changes the search algorithme.

But I agree with your point of view... grep has to be known by the users !
The "-l" flag may sometimes be preferable, for a low system for example...

----
----

Yah, if I was scripting repeated operations, by all means I'd use the more efficient way. But in general I think the modularity of *nix is actually by far the best thing about it, and I try to do things as modular as possible if there isn't a compelling reason to do otherwise. Even in my own just day-to-day stuff; sure I already know about grep, but still actually using it for everything keeps me in the habit of using it and that's more useful to me than being in the habit of using something that only applies to portversion, y'know? =)

Btw, the difference in efficiency isn't anywhere near as compelling on my systems:

ph34r# /usr/bin/time -h portversion -l '<' > /dev/null
0.88s real 0.70s user 0.18s sys
ph34r# /usr/bin/time -h portversion | grep '<' > /dev/null
0.89s real 0.68s user 0.21s sys

or

blackbox# /usr/bin/time -h portversion -l '<' > /dev/null
1.03s real 0.83s user 0.18s sys
blackbox# /usr/bin/time -h portversion | grep '<' > /dev/null
1.04s real 0.83s user 0.19s sys

On looking again at your times, I notice that the CPU% is quite a bit higher on the faster score. I have a sneaking suspicion a cron job or something kicked in while you were running the second one, so it just didn't get as much processor time and took longer.

--[[User:Jimbo|Jimbo]] 11:52, 10 Jul 2005 (EDT)

----
----

As my load average is about 1.0, I have done my tests 4 or 5 time to check that the result was always the same (it was)... But it may not have incidence on the statistics since they are updating by the kernel when the process is effectively running and collected by the '''time''' command by reading a data structure filled by the ''wait4()'' system call (If I don't remember wrong, '''time''' ''fork()'' itself, ''exec()'' the command and ''wait4()'' for it)...

What I think is that the execution time can be dependant of the number of results : As I might have everyting up to date, i might have the greater difference I can get (all the ports vs. none)...

Well... I have nothing to conclude from this ... All the two methods do the same :)
--[[User:Smortex|Smortex]]

----
----

I'm betting it has to do with your high average load. I generally work with fairly high-powered servers in relatively small environments, with load frequently averaging as low as .05 even during working times. (PS: when you're writing in talk pages, you might want to use the signature button (looks like a little squiggle up at the top of the textarea) to sign your comments, or if you're using a non-graphical browser, you can get the same effect by using <nowiki>--~~~~</nowiki> as a code.) =) --[[User:Jimbo|Jimbo]] 18:55, 10 Jul 2005 (EDT)

----
----
05/30/06
I notice that there is no mention to doing a CVS before using portupgrade; also no mention of the INDEX file being remove. Should these aspects be addressed?

----
----
absolutely... address 'em! --[[User:Jimbo|Jimbo]] 00:14, 31 May 2006 (EDT)

----
----

I just started to use FreeBSD again; so I have to do some research about it before I make a comment. Thanks Jimbo. Wed, June 7, 2006 7:51 AM CDT.

----
----

==Putting the pieces together for portupgrade==

Portupgrade depends on INDEX-6 and INDEX-6.db to function properly. As of release 5.4; INDEX-6 and INDEX-6.db are not part of the download from CVSUP. You have to build or make the INDEX* files.

1) cvsup -g -L 2 ports-supfile

2) Update the INDEX files via the three methods listed below (a,b,c):
a) cd /usr/ports, then "make index"
b) cd /usr/ports, then "makefetch index"
c) cd /usr/ports, then portsdb -Uu
3) Then upgrade your ports as described in the methods described above.

A caveat should be mentioned about the methods of obtaining the INDEX files. Option a) "make index" is officially supported by FreeBSD. Option b) is supported by FreeBSD; however, please keep in mind that the INDEX files that are retrieved from the servers may be slightly out of date; which may cause build problems. You may have to change the premission on the download INDEX files via (chmod to match the premissions: "-rw-r--r--". Option c) portsdb -Uu was never offically announced as being acceptable or not acceptable. Portsdb -Uu does create/build and update the INDEX.

My method of upgrading ports is:
Step 1, step 2c and then step 3.

The approved methods can be found here as well as the blurb about removing the INDEX files from CVSUP in section 2.5:

http://www.freebsd.org/releases/5.4R/relnotes-i386.html

--[[User:68.164.11.90|68.164.11.90]] 15:54, 2 July 2006 (EDT)Hope this helps.
Pete

: Thanks for the notes. It would be helpful to add them in to the article. The target is ''make fetchindex'' (rather than ''makefetch index''). Running ''make index'' builds an up-to-the-minute Index. You might also mention that, ''make index'' is very, very slow. <code>portsdb -U</code> creates the Index or "updates" it, but as far as I can tell it doesn't do exactly what ''make index'' does - the difference might be that the ''portsdb'' script is incrementally updating the Index (I'm not fluent in ruby - and it's a complex script); but, it isn't much faster than ''make index'', so I'm not sure except that the result is an up-to-the-minute Index, like ''make index''. There's also a port in sysutils/p5-FreeBSD-Portindex, which is a set of perl scripts that incrementally keep the Index updated, by using a cache of previous output. At least in theory, this should be faster; but I haven't used the port.

:Portupgrade runs <code>portsdb -u</code> as part of its routine before building, and between each build - so, I can't see the advantage of the extra precaution of running ''portsdb -Uu'' before running portupgrade. On the other hand, this is the procedure recommended on the [http://www.freebsd.org/cgi/man.cgi?query=portsdb&apropos=0&sektion=0&manpath=FreeBSD+6.1-RELEASE+and+Ports&format=html portsdb man page]. [[User:Ninereasons|Ninereasons]] 13:53, 4 July 2006 (EDT)

Talk:Portupgrade

2006-07-04T18:49:43Z

Ninereasons: sigh

note on the rollback: I strongly prefer showing a pipe to grep rather than the -l argument because they both accomplish the same thing, but piping to grep is a universal tool than can be used with any command or process. It's a good habit to learn using tools and skills that are universally applicable as opposed to stuff that's only applicable to one particular task, unless there is some marked advantage to using the more proprietary method of doing things.

If you know of some particularly good reason to use the -l argument instead of piping to grep, by all means let me know here. --[[User:Jimbo|Jimbo]] 19:23, 9 Jul 2005 (EDT)

----
----

The goal is just avoiding to use two processes when only one is needed... More, I tried this :
Script started on Sun Jul 10 11:35:19 2005
# time portversion -l '<' > /dev/null
portversion 1.90s user 0.68s system 86% cpu 3.001 total
# time portversion | grep '<' > /dev/null
portversion 2.68s user 0.88s system 69% cpu 5.102 total
grep '<' > /dev/null 0.00s user 0.00s system 0% cpu 5.100 total
Script done on Sun Jul 10 11:36:23 2005

I do not know how it work internaly, but I gess that the "-l" argument changes the search algorithme.

But I agree with your point of view... grep has to be known by the users !
The "-l" flag may sometimes be preferable, for a low system for example...

----
----

Yah, if I was scripting repeated operations, by all means I'd use the more efficient way. But in general I think the modularity of *nix is actually by far the best thing about it, and I try to do things as modular as possible if there isn't a compelling reason to do otherwise. Even in my own just day-to-day stuff; sure I already know about grep, but still actually using it for everything keeps me in the habit of using it and that's more useful to me than being in the habit of using something that only applies to portversion, y'know? =)

Btw, the difference in efficiency isn't anywhere near as compelling on my systems:

ph34r# /usr/bin/time -h portversion -l '<' > /dev/null
0.88s real 0.70s user 0.18s sys
ph34r# /usr/bin/time -h portversion | grep '<' > /dev/null
0.89s real 0.68s user 0.21s sys

or

blackbox# /usr/bin/time -h portversion -l '<' > /dev/null
1.03s real 0.83s user 0.18s sys
blackbox# /usr/bin/time -h portversion | grep '<' > /dev/null
1.04s real 0.83s user 0.19s sys

On looking again at your times, I notice that the CPU% is quite a bit higher on the faster score. I have a sneaking suspicion a cron job or something kicked in while you were running the second one, so it just didn't get as much processor time and took longer.

--[[User:Jimbo|Jimbo]] 11:52, 10 Jul 2005 (EDT)

----
----

As my load average is about 1.0, I have done my tests 4 or 5 time to check that the result was always the same (it was)... But it may not have incidence on the statistics since they are updating by the kernel when the process is effectively running and collected by the '''time''' command by reading a data structure filled by the ''wait4()'' system call (If I don't remember wrong, '''time''' ''fork()'' itself, ''exec()'' the command and ''wait4()'' for it)...

What I think is that the execution time can be dependant of the number of results : As I might have everyting up to date, i might have the greater difference I can get (all the ports vs. none)...

Well... I have nothing to conclude from this ... All the two methods do the same :)
--[[User:Smortex|Smortex]]

----
----

I'm betting it has to do with your high average load. I generally work with fairly high-powered servers in relatively small environments, with load frequently averaging as low as .05 even during working times. (PS: when you're writing in talk pages, you might want to use the signature button (looks like a little squiggle up at the top of the textarea) to sign your comments, or if you're using a non-graphical browser, you can get the same effect by using <nowiki>--~~~~</nowiki> as a code.) =) --[[User:Jimbo|Jimbo]] 18:55, 10 Jul 2005 (EDT)

----
----
05/30/06
I notice that there is no mention to doing a CVS before using portupgrade; also no mention of the INDEX file being remove. Should these aspects be addressed?

----
----
absolutely... address 'em! --[[User:Jimbo|Jimbo]] 00:14, 31 May 2006 (EDT)

----
----

I just started to use FreeBSD again; so I have to do some research about it before I make a comment. Thanks Jimbo. Wed, June 7, 2006 7:51 AM CDT.

----
----

==Putting the pieces together for portupgrade==

Portupgrade depends on INDEX-6 and INDEX-6.db to function properly. As of release 5.4; INDEX-6 and INDEX-6.db are not part of the download from CVSUP. You have to build or make the INDEX* files.

1) cvsup -g -L 2 ports-supfile

2) Update the INDEX files via the three methods listed below (a,b,c):
a) cd /usr/ports, then "make index"
b) cd /usr/ports, then "makefetch index"
c) cd /usr/ports, then portsdb -Uu
3) Then upgrade your ports as described in the methods described above.

A caveat should be mentioned about the methods of obtaining the INDEX files. Option a) "make index" is officially supported by FreeBSD. Option b) is supported by FreeBSD; however, please keep in mind that the INDEX files that are retrieved from the servers may be slightly out of date; which may cause build problems. You may have to change the premission on the download INDEX files via (chmod to match the premissions: "-rw-r--r--". Option c) portsdb -Uu was never offically announced as being acceptable or not acceptable. Portsdb -Uu does create/build and update the INDEX.

My method of upgrading ports is:
Step 1, step 2c and then step 3.

The approved methods can be found here as well as the blurb about removing the INDEX files from CVSUP in section 2.5:

http://www.freebsd.org/releases/5.4R/relnotes-i386.html

--[[User:68.164.11.90|68.164.11.90]] 15:54, 2 July 2006 (EDT)Hope this helps.
Pete

: Thanks for the notes. It would be helpful to add them in to the article. The target is ''make fetchindex'' (rather than ''makefetch index''). Running ''make index'' builds an up-to-the-minute Index. You might also mention that, ''make index'' is very, very slow. <code>portsdb -U</code> fetches the Index and "updates" it, but as far as I can tell it doesn't do exactly what ''make index'' does - the difference might be that the ''portsdb'' script is incrementally updating the Index (I'm not fluent in ruby - and it's a complex script); but, it isn't much faster than ''make index'', so I'm not sure except that the result is an up-to-the-minute Index, like ''make index''. There's also a port in sysutils/p5-FreeBSD-Portindex, which is a set of perl scripts that incrementally keep the Index updated, by using a cache of previous output. At least in theory, this should be faster; but I haven't used the port.

:Portupgrade runs <code>portsdb -u</code> as part of its routine before building, and between each build - so, I can't see the advantage of the extra precaution of running ''pkgdb -Uu'' before running portupgrade. [[User:Ninereasons|Ninereasons]] 13:53, 4 July 2006 (EDT)

Talk:Portupgrade

2006-07-04T18:47:37Z

Ninereasons: /* Putting the pieces together for portupgrade */

note on the rollback: I strongly prefer showing a pipe to grep rather than the -l argument because they both accomplish the same thing, but piping to grep is a universal tool than can be used with any command or process. It's a good habit to learn using tools and skills that are universally applicable as opposed to stuff that's only applicable to one particular task, unless there is some marked advantage to using the more proprietary method of doing things.

If you know of some particularly good reason to use the -l argument instead of piping to grep, by all means let me know here. --[[User:Jimbo|Jimbo]] 19:23, 9 Jul 2005 (EDT)

----
----

The goal is just avoiding to use two processes when only one is needed... More, I tried this :
Script started on Sun Jul 10 11:35:19 2005
# time portversion -l '<' > /dev/null
portversion 1.90s user 0.68s system 86% cpu 3.001 total
# time portversion | grep '<' > /dev/null
portversion 2.68s user 0.88s system 69% cpu 5.102 total
grep '<' > /dev/null 0.00s user 0.00s system 0% cpu 5.100 total
Script done on Sun Jul 10 11:36:23 2005

I do not know how it work internaly, but I gess that the "-l" argument changes the search algorithme.

But I agree with your point of view... grep has to be known by the users !
The "-l" flag may sometimes be preferable, for a low system for example...

----
----

Yah, if I was scripting repeated operations, by all means I'd use the more efficient way. But in general I think the modularity of *nix is actually by far the best thing about it, and I try to do things as modular as possible if there isn't a compelling reason to do otherwise. Even in my own just day-to-day stuff; sure I already know about grep, but still actually using it for everything keeps me in the habit of using it and that's more useful to me than being in the habit of using something that only applies to portversion, y'know? =)

Btw, the difference in efficiency isn't anywhere near as compelling on my systems:

ph34r# /usr/bin/time -h portversion -l '<' > /dev/null
0.88s real 0.70s user 0.18s sys
ph34r# /usr/bin/time -h portversion | grep '<' > /dev/null
0.89s real 0.68s user 0.21s sys

or

blackbox# /usr/bin/time -h portversion -l '<' > /dev/null
1.03s real 0.83s user 0.18s sys
blackbox# /usr/bin/time -h portversion | grep '<' > /dev/null
1.04s real 0.83s user 0.19s sys

On looking again at your times, I notice that the CPU% is quite a bit higher on the faster score. I have a sneaking suspicion a cron job or something kicked in while you were running the second one, so it just didn't get as much processor time and took longer.

--[[User:Jimbo|Jimbo]] 11:52, 10 Jul 2005 (EDT)

----
----

As my load average is about 1.0, I have done my tests 4 or 5 time to check that the result was always the same (it was)... But it may not have incidence on the statistics since they are updating by the kernel when the process is effectively running and collected by the '''time''' command by reading a data structure filled by the ''wait4()'' system call (If I don't remember wrong, '''time''' ''fork()'' itself, ''exec()'' the command and ''wait4()'' for it)...

What I think is that the execution time can be dependant of the number of results : As I might have everyting up to date, i might have the greater difference I can get (all the ports vs. none)...

Well... I have nothing to conclude from this ... All the two methods do the same :)
--[[User:Smortex|Smortex]]

----
----

I'm betting it has to do with your high average load. I generally work with fairly high-powered servers in relatively small environments, with load frequently averaging as low as .05 even during working times. (PS: when you're writing in talk pages, you might want to use the signature button (looks like a little squiggle up at the top of the textarea) to sign your comments, or if you're using a non-graphical browser, you can get the same effect by using <nowiki>--~~~~</nowiki> as a code.) =) --[[User:Jimbo|Jimbo]] 18:55, 10 Jul 2005 (EDT)

----
----
05/30/06
I notice that there is no mention to doing a CVS before using portupgrade; also no mention of the INDEX file being remove. Should these aspects be addressed?

----
----
absolutely... address 'em! --[[User:Jimbo|Jimbo]] 00:14, 31 May 2006 (EDT)

----
----

I just started to use FreeBSD again; so I have to do some research about it before I make a comment. Thanks Jimbo. Wed, June 7, 2006 7:51 AM CDT.

----
----

==Putting the pieces together for portupgrade==

Portupgrade depends on INDEX-6 and INDEX-6.db to function properly. As of release 5.4; INDEX-6 and INDEX-6.db are not part of the download from CVSUP. You have to build or make the INDEX* files.

1) cvsup -g -L 2 ports-supfile

2) Update the INDEX files via the three methods listed below (a,b,c):
a) cd /usr/ports, then "make index"
b) cd /usr/ports, then "makefetch index"
c) cd /usr/ports, then portsdb -Uu
3) Then upgrade your ports as described in the methods described above.

A caveat should be mentioned about the methods of obtaining the INDEX files. Option a) "make index" is officially supported by FreeBSD. Option b) is supported by FreeBSD; however, please keep in mind that the INDEX files that are retrieved from the servers may be slightly out of date; which may cause build problems. You may have to change the premission on the download INDEX files via (chmod to match the premissions: "-rw-r--r--". Option c) portsdb -Uu was never offically announced as being acceptable or not acceptable. Portsdb -Uu does create/build and update the INDEX.

My method of upgrading ports is:
Step 1, step 2c and then step 3.

The approved methods can be found here as well as the blurb about removing the INDEX files from CVSUP in section 2.5:

http://www.freebsd.org/releases/5.4R/relnotes-i386.html

--[[User:68.164.11.90|68.164.11.90]] 15:54, 2 July 2006 (EDT)Hope this helps.
Pete

: Thanks for the notes. It would be helpful to add them in to the article. The target is ''make fetchindex'' (rather than ''makefetch index''). Running ''make index'' builds an up-to-the-minute Index. You might also mention that, ''make index'' is very, very slow. <code>pkgdb -U</code> fetches the Index and "updates" it, but as far as I can tell it doesn't do exactly what ''make index'' does - the difference might be that the ''pkgdb'' script is incrementally updating the Index (I'm not fluent in ruby - and it's a complex script); but, it isn't much faster than ''make index'', so I'm not sure except that the result is an up-to-the-minute Index, like ''make index''. There's also a port in sysutils/p5-FreeBSD-Portindex, which is a set of perl scripts that incrementally keep the Index updated, by using a cache of previous output. At least in theory, this should be faster; but I haven't used the port.

:Portupgrade runs <code>pkgdb -u</code> as part of its routine before building, and between each build - so, I can't see the advantage of the extra precaution of running ''pkgdb -Uu'' before running portupgrade. [[User:Ninereasons|Ninereasons]] 13:53, 4 July 2006 (EDT)

Talk:Portupgrade

2006-07-04T18:47:11Z

Ninereasons: /* Putting the pieces together for portupgrade */

note on the rollback: I strongly prefer showing a pipe to grep rather than the -l argument because they both accomplish the same thing, but piping to grep is a universal tool than can be used with any command or process. It's a good habit to learn using tools and skills that are universally applicable as opposed to stuff that's only applicable to one particular task, unless there is some marked advantage to using the more proprietary method of doing things.

If you know of some particularly good reason to use the -l argument instead of piping to grep, by all means let me know here. --[[User:Jimbo|Jimbo]] 19:23, 9 Jul 2005 (EDT)

----
----

The goal is just avoiding to use two processes when only one is needed... More, I tried this :
Script started on Sun Jul 10 11:35:19 2005
# time portversion -l '<' > /dev/null
portversion 1.90s user 0.68s system 86% cpu 3.001 total
# time portversion | grep '<' > /dev/null
portversion 2.68s user 0.88s system 69% cpu 5.102 total
grep '<' > /dev/null 0.00s user 0.00s system 0% cpu 5.100 total
Script done on Sun Jul 10 11:36:23 2005

I do not know how it work internaly, but I gess that the "-l" argument changes the search algorithme.

But I agree with your point of view... grep has to be known by the users !
The "-l" flag may sometimes be preferable, for a low system for example...

----
----

Yah, if I was scripting repeated operations, by all means I'd use the more efficient way. But in general I think the modularity of *nix is actually by far the best thing about it, and I try to do things as modular as possible if there isn't a compelling reason to do otherwise. Even in my own just day-to-day stuff; sure I already know about grep, but still actually using it for everything keeps me in the habit of using it and that's more useful to me than being in the habit of using something that only applies to portversion, y'know? =)

Btw, the difference in efficiency isn't anywhere near as compelling on my systems:

ph34r# /usr/bin/time -h portversion -l '<' > /dev/null
0.88s real 0.70s user 0.18s sys
ph34r# /usr/bin/time -h portversion | grep '<' > /dev/null
0.89s real 0.68s user 0.21s sys

or

blackbox# /usr/bin/time -h portversion -l '<' > /dev/null
1.03s real 0.83s user 0.18s sys
blackbox# /usr/bin/time -h portversion | grep '<' > /dev/null
1.04s real 0.83s user 0.19s sys

On looking again at your times, I notice that the CPU% is quite a bit higher on the faster score. I have a sneaking suspicion a cron job or something kicked in while you were running the second one, so it just didn't get as much processor time and took longer.

--[[User:Jimbo|Jimbo]] 11:52, 10 Jul 2005 (EDT)

----
----

As my load average is about 1.0, I have done my tests 4 or 5 time to check that the result was always the same (it was)... But it may not have incidence on the statistics since they are updating by the kernel when the process is effectively running and collected by the '''time''' command by reading a data structure filled by the ''wait4()'' system call (If I don't remember wrong, '''time''' ''fork()'' itself, ''exec()'' the command and ''wait4()'' for it)...

What I think is that the execution time can be dependant of the number of results : As I might have everyting up to date, i might have the greater difference I can get (all the ports vs. none)...

Well... I have nothing to conclude from this ... All the two methods do the same :)
--[[User:Smortex|Smortex]]

----
----

I'm betting it has to do with your high average load. I generally work with fairly high-powered servers in relatively small environments, with load frequently averaging as low as .05 even during working times. (PS: when you're writing in talk pages, you might want to use the signature button (looks like a little squiggle up at the top of the textarea) to sign your comments, or if you're using a non-graphical browser, you can get the same effect by using <nowiki>--~~~~</nowiki> as a code.) =) --[[User:Jimbo|Jimbo]] 18:55, 10 Jul 2005 (EDT)

----
----
05/30/06
I notice that there is no mention to doing a CVS before using portupgrade; also no mention of the INDEX file being remove. Should these aspects be addressed?

----
----
absolutely... address 'em! --[[User:Jimbo|Jimbo]] 00:14, 31 May 2006 (EDT)

----
----

I just started to use FreeBSD again; so I have to do some research about it before I make a comment. Thanks Jimbo. Wed, June 7, 2006 7:51 AM CDT.

----
----

==Putting the pieces together for portupgrade==

Portupgrade depends on INDEX-6 and INDEX-6.db to function properly. As of release 5.4; INDEX-6 and INDEX-6.db are not part of the download from CVSUP. You have to build or make the INDEX* files.

1) cvsup -g -L 2 ports-supfile

2) Update the INDEX files via the three methods listed below (a,b,c):
a) cd /usr/ports, then "make index"
b) cd /usr/ports, then "makefetch index"
c) cd /usr/ports, then portsdb -Uu
3) Then upgrade your ports as described in the methods described above.

A caveat should be mentioned about the methods of obtaining the INDEX files. Option a) "make index" is officially supported by FreeBSD. Option b) is supported by FreeBSD; however, please keep in mind that the INDEX files that are retrieved from the servers may be slightly out of date; which may cause build problems. You may have to change the premission on the download INDEX files via (chmod to match the premissions: "-rw-r--r--". Option c) portsdb -Uu was never offically announced as being acceptable or not acceptable. Portsdb -Uu does create/build and update the INDEX.

My method of upgrading ports is:
Step 1, step 2c and then step 3.

The approved methods can be found here as well as the blurb about removing the INDEX files from CVSUP in section 2.5:

http://www.freebsd.org/releases/5.4R/relnotes-i386.html

--[[User:68.164.11.90|68.164.11.90]] 15:54, 2 July 2006 (EDT)Hope this helps.
Pete

: Thanks for the notes. It would be helpful to add them in to the article. The target is ''make fetchindex'' (rather than ''makefetch index''). Running ''make index'' builds an up-to-the-minute Index. You might also mention that, ''make index'' is very, very slow. <code>pkgdb -U</code> fetches the Index and "updates" it, but as far as I can tell it doesn't do exactly what ''make index'' does - the difference might be that the ''portsdb'' script is incrementally updating the Index (I'm not fluent in ruby - and it's a complex script); but, it isn't much faster than ''make index'', so I'm not sure except that the result is an up-to-the-minute Index, like ''make index''. There's also a port in sysutils/p5-FreeBSD-Portindex, which is a set of perl scripts that incrementally keep the Index updated, by using a cache of previous output. At least in theory, this should be faster; but I haven't used the port.

:Portupgrade runs <code>pkgdb -u</code> as part of its routine before building, and between each build - so, I can't see the advantage of the extra precaution of running ''pkgdb -Uu'' before running portupgrade. [[User:Ninereasons|Ninereasons]] 13:53, 4 July 2006 (EDT)

Talk:Portupgrade

2006-07-04T18:45:42Z

Ninereasons: /* Putting the pieces together for portupgrade */

note on the rollback: I strongly prefer showing a pipe to grep rather than the -l argument because they both accomplish the same thing, but piping to grep is a universal tool than can be used with any command or process. It's a good habit to learn using tools and skills that are universally applicable as opposed to stuff that's only applicable to one particular task, unless there is some marked advantage to using the more proprietary method of doing things.

If you know of some particularly good reason to use the -l argument instead of piping to grep, by all means let me know here. --[[User:Jimbo|Jimbo]] 19:23, 9 Jul 2005 (EDT)

----
----

The goal is just avoiding to use two processes when only one is needed... More, I tried this :
Script started on Sun Jul 10 11:35:19 2005
# time portversion -l '<' > /dev/null
portversion 1.90s user 0.68s system 86% cpu 3.001 total
# time portversion | grep '<' > /dev/null
portversion 2.68s user 0.88s system 69% cpu 5.102 total
grep '<' > /dev/null 0.00s user 0.00s system 0% cpu 5.100 total
Script done on Sun Jul 10 11:36:23 2005

I do not know how it work internaly, but I gess that the "-l" argument changes the search algorithme.

But I agree with your point of view... grep has to be known by the users !
The "-l" flag may sometimes be preferable, for a low system for example...

----
----

Yah, if I was scripting repeated operations, by all means I'd use the more efficient way. But in general I think the modularity of *nix is actually by far the best thing about it, and I try to do things as modular as possible if there isn't a compelling reason to do otherwise. Even in my own just day-to-day stuff; sure I already know about grep, but still actually using it for everything keeps me in the habit of using it and that's more useful to me than being in the habit of using something that only applies to portversion, y'know? =)

Btw, the difference in efficiency isn't anywhere near as compelling on my systems:

ph34r# /usr/bin/time -h portversion -l '<' > /dev/null
0.88s real 0.70s user 0.18s sys
ph34r# /usr/bin/time -h portversion | grep '<' > /dev/null
0.89s real 0.68s user 0.21s sys

or

blackbox# /usr/bin/time -h portversion -l '<' > /dev/null
1.03s real 0.83s user 0.18s sys
blackbox# /usr/bin/time -h portversion | grep '<' > /dev/null
1.04s real 0.83s user 0.19s sys

On looking again at your times, I notice that the CPU% is quite a bit higher on the faster score. I have a sneaking suspicion a cron job or something kicked in while you were running the second one, so it just didn't get as much processor time and took longer.

--[[User:Jimbo|Jimbo]] 11:52, 10 Jul 2005 (EDT)

----
----

As my load average is about 1.0, I have done my tests 4 or 5 time to check that the result was always the same (it was)... But it may not have incidence on the statistics since they are updating by the kernel when the process is effectively running and collected by the '''time''' command by reading a data structure filled by the ''wait4()'' system call (If I don't remember wrong, '''time''' ''fork()'' itself, ''exec()'' the command and ''wait4()'' for it)...

What I think is that the execution time can be dependant of the number of results : As I might have everyting up to date, i might have the greater difference I can get (all the ports vs. none)...

Well... I have nothing to conclude from this ... All the two methods do the same :)
--[[User:Smortex|Smortex]]

----
----

I'm betting it has to do with your high average load. I generally work with fairly high-powered servers in relatively small environments, with load frequently averaging as low as .05 even during working times. (PS: when you're writing in talk pages, you might want to use the signature button (looks like a little squiggle up at the top of the textarea) to sign your comments, or if you're using a non-graphical browser, you can get the same effect by using <nowiki>--~~~~</nowiki> as a code.) =) --[[User:Jimbo|Jimbo]] 18:55, 10 Jul 2005 (EDT)

----
----
05/30/06
I notice that there is no mention to doing a CVS before using portupgrade; also no mention of the INDEX file being remove. Should these aspects be addressed?

----
----
absolutely... address 'em! --[[User:Jimbo|Jimbo]] 00:14, 31 May 2006 (EDT)

----
----

I just started to use FreeBSD again; so I have to do some research about it before I make a comment. Thanks Jimbo. Wed, June 7, 2006 7:51 AM CDT.

----
----

==Putting the pieces together for portupgrade==

Portupgrade depends on INDEX-6 and INDEX-6.db to function properly. As of release 5.4; INDEX-6 and INDEX-6.db are not part of the download from CVSUP. You have to build or make the INDEX* files.

1) cvsup -g -L 2 ports-supfile

2) Update the INDEX files via the three methods listed below (a,b,c):
a) cd /usr/ports, then "make index"
b) cd /usr/ports, then "makefetch index"
c) cd /usr/ports, then portsdb -Uu
3) Then upgrade your ports as described in the methods described above.

A caveat should be mentioned about the methods of obtaining the INDEX files. Option a) "make index" is officially supported by FreeBSD. Option b) is supported by FreeBSD; however, please keep in mind that the INDEX files that are retrieved from the servers may be slightly out of date; which may cause build problems. You may have to change the premission on the download INDEX files via (chmod to match the premissions: "-rw-r--r--". Option c) portsdb -Uu was never offically announced as being acceptable or not acceptable. Portsdb -Uu does create/build and update the INDEX.

My method of upgrading ports is:
Step 1, step 2c and then step 3.

The approved methods can be found here as well as the blurb about removing the INDEX files from CVSUP in section 2.5:

http://www.freebsd.org/releases/5.4R/relnotes-i386.html

--[[User:68.164.11.90|68.164.11.90]] 15:54, 2 July 2006 (EDT)Hope this helps.
Pete

: Thanks for the notes. It would be helpful to add them in to the article. The target is ''make fetchindex'' (rather than ''makefetch index''). Running ''make index'' builds an up-to-the-minute Index. You might also mention that, ''make index'' is very, very slow. <code>portsdb -U</code> fetches the Index and "updates" it, but as far as I can tell it doesn't do exactly what ''make index'' does - the difference might be that the ''portsdb'' script is incrementally updating the Index (I'm not fluent in ruby - and it's a complex script); but, it isn't much faster than ''make index'', so I'm not sure except that the result is an up-to-the-minute Index, like ''make index''. There's also a port in sysutils/p5-FreeBSD-Portindex, which is a set of perl scripts that incrementally keep the Index updated, by using a cache of previous output. At least in theory, this should be faster; but I haven't used the port.

:Portupgrade runs <code>portsdb -u</code> as part of its routine before building, and between each build - so, I can't see the advantage of the extra precaution of running ''pkgdb -Uu'' before running portupgrade. [[User:Ninereasons|Ninereasons]] 13:53, 4 July 2006 (EDT)

Talk:Portupgrade

2006-07-04T18:16:43Z

Ninereasons: /* Putting the pieces together for portupgrade */

note on the rollback: I strongly prefer showing a pipe to grep rather than the -l argument because they both accomplish the same thing, but piping to grep is a universal tool than can be used with any command or process. It's a good habit to learn using tools and skills that are universally applicable as opposed to stuff that's only applicable to one particular task, unless there is some marked advantage to using the more proprietary method of doing things.

If you know of some particularly good reason to use the -l argument instead of piping to grep, by all means let me know here. --[[User:Jimbo|Jimbo]] 19:23, 9 Jul 2005 (EDT)

----
----

The goal is just avoiding to use two processes when only one is needed... More, I tried this :
Script started on Sun Jul 10 11:35:19 2005
# time portversion -l '<' > /dev/null
portversion 1.90s user 0.68s system 86% cpu 3.001 total
# time portversion | grep '<' > /dev/null
portversion 2.68s user 0.88s system 69% cpu 5.102 total
grep '<' > /dev/null 0.00s user 0.00s system 0% cpu 5.100 total
Script done on Sun Jul 10 11:36:23 2005

I do not know how it work internaly, but I gess that the "-l" argument changes the search algorithme.

But I agree with your point of view... grep has to be known by the users !
The "-l" flag may sometimes be preferable, for a low system for example...

----
----

Yah, if I was scripting repeated operations, by all means I'd use the more efficient way. But in general I think the modularity of *nix is actually by far the best thing about it, and I try to do things as modular as possible if there isn't a compelling reason to do otherwise. Even in my own just day-to-day stuff; sure I already know about grep, but still actually using it for everything keeps me in the habit of using it and that's more useful to me than being in the habit of using something that only applies to portversion, y'know? =)

Btw, the difference in efficiency isn't anywhere near as compelling on my systems:

ph34r# /usr/bin/time -h portversion -l '<' > /dev/null
0.88s real 0.70s user 0.18s sys
ph34r# /usr/bin/time -h portversion | grep '<' > /dev/null
0.89s real 0.68s user 0.21s sys

or

blackbox# /usr/bin/time -h portversion -l '<' > /dev/null
1.03s real 0.83s user 0.18s sys
blackbox# /usr/bin/time -h portversion | grep '<' > /dev/null
1.04s real 0.83s user 0.19s sys

On looking again at your times, I notice that the CPU% is quite a bit higher on the faster score. I have a sneaking suspicion a cron job or something kicked in while you were running the second one, so it just didn't get as much processor time and took longer.

--[[User:Jimbo|Jimbo]] 11:52, 10 Jul 2005 (EDT)

----
----

As my load average is about 1.0, I have done my tests 4 or 5 time to check that the result was always the same (it was)... But it may not have incidence on the statistics since they are updating by the kernel when the process is effectively running and collected by the '''time''' command by reading a data structure filled by the ''wait4()'' system call (If I don't remember wrong, '''time''' ''fork()'' itself, ''exec()'' the command and ''wait4()'' for it)...

What I think is that the execution time can be dependant of the number of results : As I might have everyting up to date, i might have the greater difference I can get (all the ports vs. none)...

Well... I have nothing to conclude from this ... All the two methods do the same :)
--[[User:Smortex|Smortex]]

----
----

I'm betting it has to do with your high average load. I generally work with fairly high-powered servers in relatively small environments, with load frequently averaging as low as .05 even during working times. (PS: when you're writing in talk pages, you might want to use the signature button (looks like a little squiggle up at the top of the textarea) to sign your comments, or if you're using a non-graphical browser, you can get the same effect by using <nowiki>--~~~~</nowiki> as a code.) =) --[[User:Jimbo|Jimbo]] 18:55, 10 Jul 2005 (EDT)

----
----
05/30/06
I notice that there is no mention to doing a CVS before using portupgrade; also no mention of the INDEX file being remove. Should these aspects be addressed?

----
----
absolutely... address 'em! --[[User:Jimbo|Jimbo]] 00:14, 31 May 2006 (EDT)

----
----

I just started to use FreeBSD again; so I have to do some research about it before I make a comment. Thanks Jimbo. Wed, June 7, 2006 7:51 AM CDT.

----
----

==Putting the pieces together for portupgrade==

Portupgrade depends on INDEX-6 and INDEX-6.db to function properly. As of release 5.4; INDEX-6 and INDEX-6.db are not part of the download from CVSUP. You have to build or make the INDEX* files.

1) cvsup -g -L 2 ports-supfile

2) Update the INDEX files via the three methods listed below (a,b,c):
a) cd /usr/ports, then "make index"
b) cd /usr/ports, then "makefetch index"
c) cd /usr/ports, then portsdb -Uu
3) Then upgrade your ports as described in the methods described above.

A caveat should be mentioned about the methods of obtaining the INDEX files. Option a) "make index" is officially supported by FreeBSD. Option b) is supported by FreeBSD; however, please keep in mind that the INDEX files that are retrieved from the servers may be slightly out of date; which may cause build problems. You may have to change the premission on the download INDEX files via (chmod to match the premissions: "-rw-r--r--". Option c) portsdb -Uu was never offically announced as being acceptable or not acceptable. Portsdb -Uu does create/build and update the INDEX.

My method of upgrading ports is:
Step 1, step 2c and then step 3.

The approved methods can be found here as well as the blurb about removing the INDEX files from CVSUP in section 2.5:

http://www.freebsd.org/releases/5.4R/relnotes-i386.html

--[[User:68.164.11.90|68.164.11.90]] 15:54, 2 July 2006 (EDT)Hope this helps.
Pete

: Thanks for the notes. It would be helpful to add them in to the article. The target is ''make fetchindex'' (rather than ''makefetch index''). Running ''make index'' is the only way to build an up-to-the-minute Index. You might also mention that, ''make index'' is very, very slow. <code>portsdb -U</code> fetches the Index, rather than building it. Portupgrade runs <code>portsdb -u</code> as part of its routine before building, and between each build. There's also a port in sysutils/p5-FreeBSD-Portindex, which is a set of perl scripts that make it possible to update the Index incrementally. [[User:Ninereasons|Ninereasons]] 13:53, 4 July 2006 (EDT)

Talk:Portupgrade

2006-07-04T18:09:11Z

Ninereasons: /* Putting the pieces together for portupgrade */

note on the rollback: I strongly prefer showing a pipe to grep rather than the -l argument because they both accomplish the same thing, but piping to grep is a universal tool than can be used with any command or process. It's a good habit to learn using tools and skills that are universally applicable as opposed to stuff that's only applicable to one particular task, unless there is some marked advantage to using the more proprietary method of doing things.

If you know of some particularly good reason to use the -l argument instead of piping to grep, by all means let me know here. --[[User:Jimbo|Jimbo]] 19:23, 9 Jul 2005 (EDT)

----
----

The goal is just avoiding to use two processes when only one is needed... More, I tried this :
Script started on Sun Jul 10 11:35:19 2005
# time portversion -l '<' > /dev/null
portversion 1.90s user 0.68s system 86% cpu 3.001 total
# time portversion | grep '<' > /dev/null
portversion 2.68s user 0.88s system 69% cpu 5.102 total
grep '<' > /dev/null 0.00s user 0.00s system 0% cpu 5.100 total
Script done on Sun Jul 10 11:36:23 2005

I do not know how it work internaly, but I gess that the "-l" argument changes the search algorithme.

But I agree with your point of view... grep has to be known by the users !
The "-l" flag may sometimes be preferable, for a low system for example...

----
----

Yah, if I was scripting repeated operations, by all means I'd use the more efficient way. But in general I think the modularity of *nix is actually by far the best thing about it, and I try to do things as modular as possible if there isn't a compelling reason to do otherwise. Even in my own just day-to-day stuff; sure I already know about grep, but still actually using it for everything keeps me in the habit of using it and that's more useful to me than being in the habit of using something that only applies to portversion, y'know? =)

Btw, the difference in efficiency isn't anywhere near as compelling on my systems:

ph34r# /usr/bin/time -h portversion -l '<' > /dev/null
0.88s real 0.70s user 0.18s sys
ph34r# /usr/bin/time -h portversion | grep '<' > /dev/null
0.89s real 0.68s user 0.21s sys

or

blackbox# /usr/bin/time -h portversion -l '<' > /dev/null
1.03s real 0.83s user 0.18s sys
blackbox# /usr/bin/time -h portversion | grep '<' > /dev/null
1.04s real 0.83s user 0.19s sys

On looking again at your times, I notice that the CPU% is quite a bit higher on the faster score. I have a sneaking suspicion a cron job or something kicked in while you were running the second one, so it just didn't get as much processor time and took longer.

--[[User:Jimbo|Jimbo]] 11:52, 10 Jul 2005 (EDT)

----
----

As my load average is about 1.0, I have done my tests 4 or 5 time to check that the result was always the same (it was)... But it may not have incidence on the statistics since they are updating by the kernel when the process is effectively running and collected by the '''time''' command by reading a data structure filled by the ''wait4()'' system call (If I don't remember wrong, '''time''' ''fork()'' itself, ''exec()'' the command and ''wait4()'' for it)...

What I think is that the execution time can be dependant of the number of results : As I might have everyting up to date, i might have the greater difference I can get (all the ports vs. none)...

Well... I have nothing to conclude from this ... All the two methods do the same :)
--[[User:Smortex|Smortex]]

----
----

I'm betting it has to do with your high average load. I generally work with fairly high-powered servers in relatively small environments, with load frequently averaging as low as .05 even during working times. (PS: when you're writing in talk pages, you might want to use the signature button (looks like a little squiggle up at the top of the textarea) to sign your comments, or if you're using a non-graphical browser, you can get the same effect by using <nowiki>--~~~~</nowiki> as a code.) =) --[[User:Jimbo|Jimbo]] 18:55, 10 Jul 2005 (EDT)

----
----
05/30/06
I notice that there is no mention to doing a CVS before using portupgrade; also no mention of the INDEX file being remove. Should these aspects be addressed?

----
----
absolutely... address 'em! --[[User:Jimbo|Jimbo]] 00:14, 31 May 2006 (EDT)

----
----

I just started to use FreeBSD again; so I have to do some research about it before I make a comment. Thanks Jimbo. Wed, June 7, 2006 7:51 AM CDT.

----
----

==Putting the pieces together for portupgrade==

Portupgrade depends on INDEX-6 and INDEX-6.db to function properly. As of release 5.4; INDEX-6 and INDEX-6.db are not part of the download from CVSUP. You have to build or make the INDEX* files.

1) cvsup -g -L 2 ports-supfile

2) Update the INDEX files via the three methods listed below (a,b,c):
a) cd /usr/ports, then "make index"
b) cd /usr/ports, then "makefetch index"
c) cd /usr/ports, then portsdb -Uu
3) Then upgrade your ports as described in the methods described above.

A caveat should be mentioned about the methods of obtaining the INDEX files. Option a) "make index" is officially supported by FreeBSD. Option b) is supported by FreeBSD; however, please keep in mind that the INDEX files that are retrieved from the servers may be slightly out of date; which may cause build problems. You may have to change the premission on the download INDEX files via (chmod to match the premissions: "-rw-r--r--". Option c) portsdb -Uu was never offically announced as being acceptable or not acceptable. Portsdb -Uu does create/build and update the INDEX.

My method of upgrading ports is:
Step 1, step 2c and then step 3.

The approved methods can be found here as well as the blurb about removing the INDEX files from CVSUP in section 2.5:

http://www.freebsd.org/releases/5.4R/relnotes-i386.html

--[[User:68.164.11.90|68.164.11.90]] 15:54, 2 July 2006 (EDT)Hope this helps.
Pete

: Thanks for the notes. It would be helpful to add them in to the article. The target is ''make fetchindex'' (rather than ''makefetch index''). Portupgrade runs <code>portsdb -u</code> as part of its routine before building, and between each build. You might also mention that, ''make index'' is very, very slow. There's also a port in sysutils/p5-FreeBSD-Portindex, which is a set of perl scripts that make it possible to update the Index incrementally. [[User:Ninereasons|Ninereasons]] 13:53, 4 July 2006 (EDT)

Talk:Portupgrade

2006-07-04T17:59:13Z

Ninereasons: /* Putting the pieces together for portupgrade */

note on the rollback: I strongly prefer showing a pipe to grep rather than the -l argument because they both accomplish the same thing, but piping to grep is a universal tool than can be used with any command or process. It's a good habit to learn using tools and skills that are universally applicable as opposed to stuff that's only applicable to one particular task, unless there is some marked advantage to using the more proprietary method of doing things.

If you know of some particularly good reason to use the -l argument instead of piping to grep, by all means let me know here. --[[User:Jimbo|Jimbo]] 19:23, 9 Jul 2005 (EDT)

----
----

The goal is just avoiding to use two processes when only one is needed... More, I tried this :
Script started on Sun Jul 10 11:35:19 2005
# time portversion -l '<' > /dev/null
portversion 1.90s user 0.68s system 86% cpu 3.001 total
# time portversion | grep '<' > /dev/null
portversion 2.68s user 0.88s system 69% cpu 5.102 total
grep '<' > /dev/null 0.00s user 0.00s system 0% cpu 5.100 total
Script done on Sun Jul 10 11:36:23 2005

I do not know how it work internaly, but I gess that the "-l" argument changes the search algorithme.

But I agree with your point of view... grep has to be known by the users !
The "-l" flag may sometimes be preferable, for a low system for example...

----
----

Yah, if I was scripting repeated operations, by all means I'd use the more efficient way. But in general I think the modularity of *nix is actually by far the best thing about it, and I try to do things as modular as possible if there isn't a compelling reason to do otherwise. Even in my own just day-to-day stuff; sure I already know about grep, but still actually using it for everything keeps me in the habit of using it and that's more useful to me than being in the habit of using something that only applies to portversion, y'know? =)

Btw, the difference in efficiency isn't anywhere near as compelling on my systems:

ph34r# /usr/bin/time -h portversion -l '<' > /dev/null
0.88s real 0.70s user 0.18s sys
ph34r# /usr/bin/time -h portversion | grep '<' > /dev/null
0.89s real 0.68s user 0.21s sys

or

blackbox# /usr/bin/time -h portversion -l '<' > /dev/null
1.03s real 0.83s user 0.18s sys
blackbox# /usr/bin/time -h portversion | grep '<' > /dev/null
1.04s real 0.83s user 0.19s sys

On looking again at your times, I notice that the CPU% is quite a bit higher on the faster score. I have a sneaking suspicion a cron job or something kicked in while you were running the second one, so it just didn't get as much processor time and took longer.

--[[User:Jimbo|Jimbo]] 11:52, 10 Jul 2005 (EDT)

----
----

As my load average is about 1.0, I have done my tests 4 or 5 time to check that the result was always the same (it was)... But it may not have incidence on the statistics since they are updating by the kernel when the process is effectively running and collected by the '''time''' command by reading a data structure filled by the ''wait4()'' system call (If I don't remember wrong, '''time''' ''fork()'' itself, ''exec()'' the command and ''wait4()'' for it)...

What I think is that the execution time can be dependant of the number of results : As I might have everyting up to date, i might have the greater difference I can get (all the ports vs. none)...

Well... I have nothing to conclude from this ... All the two methods do the same :)
--[[User:Smortex|Smortex]]

----
----

I'm betting it has to do with your high average load. I generally work with fairly high-powered servers in relatively small environments, with load frequently averaging as low as .05 even during working times. (PS: when you're writing in talk pages, you might want to use the signature button (looks like a little squiggle up at the top of the textarea) to sign your comments, or if you're using a non-graphical browser, you can get the same effect by using <nowiki>--~~~~</nowiki> as a code.) =) --[[User:Jimbo|Jimbo]] 18:55, 10 Jul 2005 (EDT)

----
----
05/30/06
I notice that there is no mention to doing a CVS before using portupgrade; also no mention of the INDEX file being remove. Should these aspects be addressed?

----
----
absolutely... address 'em! --[[User:Jimbo|Jimbo]] 00:14, 31 May 2006 (EDT)

----
----

I just started to use FreeBSD again; so I have to do some research about it before I make a comment. Thanks Jimbo. Wed, June 7, 2006 7:51 AM CDT.

----
----

==Putting the pieces together for portupgrade==

Portupgrade depends on INDEX-6 and INDEX-6.db to function properly. As of release 5.4; INDEX-6 and INDEX-6.db are not part of the download from CVSUP. You have to build or make the INDEX* files.

1) cvsup -g -L 2 ports-supfile

2) Update the INDEX files via the three methods listed below (a,b,c):
a) cd /usr/ports, then "make index"
b) cd /usr/ports, then "makefetch index"
c) cd /usr/ports, then portsdb -Uu
3) Then upgrade your ports as described in the methods described above.

A caveat should be mentioned about the methods of obtaining the INDEX files. Option a) "make index" is officially supported by FreeBSD. Option b) is supported by FreeBSD; however, please keep in mind that the INDEX files that are retrieved from the servers may be slightly out of date; which may cause build problems. You may have to change the premission on the download INDEX files via (chmod to match the premissions: "-rw-r--r--". Option c) portsdb -Uu was never offically announced as being acceptable or not acceptable. Portsdb -Uu does create/build and update the INDEX.

My method of upgrading ports is:
Step 1, step 2c and then step 3.

The approved methods can be found here as well as the blurb about removing the INDEX files from CVSUP in section 2.5:

http://www.freebsd.org/releases/5.4R/relnotes-i386.html

--[[User:68.164.11.90|68.164.11.90]] 15:54, 2 July 2006 (EDT)Hope this helps.
Pete

: Thanks for the notes. It would be helpful to add them in to the article. The target is ''make fetchindex'' (rather than ''makefetch index''). Portupgrade runs <code>portsdb -u</code> as part of its routine before building, and between each build. You might also mention that, ''make index'' is very, very slow. [[User:Ninereasons|Ninereasons]] 13:53, 4 July 2006 (EDT)

Talk:Portupgrade

2006-07-04T17:53:52Z

Ninereasons: /* Putting the pieces together for portupgrade */ - the target is 'fetchindex'

Redirection

2006-06-30T01:28:34Z

Ninereasons: change bullets to sub-heads

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Shell redirection operators include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipelines and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). When the next process inherits the open ''fd'' from the shell the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

=== ''tcsh'' and ''sh'' ===
# Write output to a file
#: % mycmd > out.txt
#:: stderr stuff
# Append output to a file
#: % mycmd >> out.txt
#:: stderr stuff
# Redirect the output of a remote command to ''local.txt''.
#: % localcmd "remotecmd" > local.txt
#:: stderr stuff
# Same command as above, showing only the changes compared to ''local.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin'. These two commands are equivalent. ''
#: % localcmd "remotecmd" | diff /dev/stdin local.txt
#: % localcmd "remotecmd" | diff - local.txt
#:: stderr stuff
# Direct <code>stdout</code>+<code>stderr</code> to file
#: % mycmd >& out.txt
# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
#: % <jumble.txt sort >sorted.txt
#: % >sorted.txt sort <jumble.txt
#: % sort < jumble.txt >sorted.txt
#: % <jumble.txt>sorted.txt sort
#:: stderr stuff
# Sort unique lines of jumble.txt into sorted.txt
#: % <jumble.txt sort | uniq >sorted.txt
#:: stderr stuff
# Sort ''HERE doc'' delimited by "lines"
#: % <<lines sort 
#::? a second line 
#::? a first line 
#::lines a first line a second line stderr stuff

=== ''tcsh'' only ===
# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
#: % ( mycmd > /dev/stdin ) > & /dev/null
#:: stdout stuff
#: % ( mycmd > /dev/tty ) > & /dev/null
#:: stdout stuff
# Append output to out.txt; discard messages
#: % (mycmd >> out.txt) >& /dev/null
# Write output to out.txt; store and watch errors ''Note: Compare the same task in ''sh''. ''
#: % ( mycmd > out.txt ) | & tee err.txt
#:: stderr stuff

=== ''sh'' only ===
# Discard errors, watch output
#: $ mycmd 2> /dev/null
#:: stdout stuff
# Append output to out.txt; discard messages
#: $ mycmd 2> /dev/null >> out.txt
# Write output to out.txt; store and watch errors ''Note: Compare the same task in ''tcsh''. ''
#: $ mycmd 2>&1 > out.txt | tee err.txt
#:: stderr stuff
# Write messages to err.txt; write output to out.txt and copy output to terminal
#: $ mycmd 2> err.txt | tee out.txt
#:: stdout stuff
# Assign a variable from stored.txt
#: $ <stored.txt read var; mycmd $var
#:: stderr stuff
#:: stdout stuff
# Assign first three lines of stored.txt to three different variables
#: $ exec 3<&0; exec <stored.txt; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
#:: stderr stuff
#:: stdout stuff
# Use all unique lines in stored.txt as an argument variable for ''mycmd'', appending to result.txt
#: $ exec 3<&0; exec <stored.txt; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
#:: stderr stuff
: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
:: $ mycmd 1> out.txt
:: $ mycmd > out.txt
::: stderr stuff
: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
:: $ mycmd 0< in.txt
:: $ mycmd < in.txt
::: stderr stuff
::: stdout stuff
: ''Note:'' To close a file descriptor, say <code>n<&-</code>
:: $ mycmd >out.txt 1<&-;
:: ''-sh: fails without messages''
:: -bash: mycmd: write error: Bad file descriptor

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to another command. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 | command > errors.txt
because when you switched output to the messages file descriptor, it's done ''right away''. When you tried to switch errors to output there's nothing left to pipe. It's exactly as though you had said
samizdata# myprogram 1>/dev/stderr | command > errors.txt
''command'' receives no data, because it's all been sent to your screen. ''errors.txt'' will be empty. If you had tried it the other way, it's just as though you would have said:
samizdata# myprogram 2>/dev/stdout | command > errors.txt
In which case, errors and output are both piped to ''command'', which was not what you wanted.

This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, like this:
samizdata# myprogram 3>&2 2>&1 1>&3 | command > errors.txt

Let's step through what this does.
# ''fd 3'' points to <code>stderr</code>
# ''fd 2'' points to <code>stdout</code>
# ''fd 1'' points to ''fd 3'' (which is connected to <code>stderr</code>)
# ''|'' passes the open file descriptor connected to <code>stdout</code> (''fd 2'') making it <code>stdin</code> for ''command''
# ''>'' redirects the output of ''command'' to ''errors.txt''.

So, what you'll see on your screen is the output of ''myprogram'' on <code>stderr</code> (not passed to ''command''), followed by errors (if any) from ''command''. In short, what's been accomplished is, a swap of errors for output, from ''myprogram'' to ''command'', the output of which is redirected to ''errors.txt''. There is no reliable way to do the same thing in <code>tcsh</code>.

Also, see the [http://www.freebsd.org/cgi/man.cgi?query=mkfifo&apropos=0&sektion=1&manpath=FreeBSD+6.1-RELEASE&format=html man page] for [[mkfifo]], a utility for creating arbitrary file descriptor files.
[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Redirection

2006-06-28T17:08:44Z

Ninereasons: operand (noun) => operator(verb)

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Shell redirection operators include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipelines and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). When the next process inherits the open ''fd'' from the shell the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

* '''tcsh and sh'''
*# Write output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Append output to a file
*#: % mycmd >> out.txt
*#:: stderr stuff
*# Redirect the output of a remote command to ''local.txt''.
*#: % localcmd "remotecmd" > local.txt
*#:: stderr stuff
*# Same command as above, showing only the changes compared to ''local.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin'. These two commands are equivalent. ''
*#: % localcmd "remotecmd" | diff /dev/stdin local.txt
*#: % localcmd "remotecmd" | diff - local.txt
*#:: stderr stuff
*# Direct <code>stdout</code>+<code>stderr</code> to file
*#: % mycmd >& out.txt
*# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
*#: % <jumble.txt sort >sorted.txt
*#: % >sorted.txt sort <jumble.txt
*#: % sort < jumble.txt >sorted.txt
*#: % <jumble.txt>sorted.txt sort
*#:: stderr stuff
*# Sort unique lines of jumble.txt into sorted.txt
*#: % <jumble.txt sort | uniq >sorted.txt
*#:: stderr stuff
*# Sort ''HERE doc'' delimited by "lines"
*#: % <<lines sort 
*#::? a second line 
*#::? a first line 
*#::lines a first line a second line stderr stuff

* '''tcsh''' only
*# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
*#: % ( mycmd > /dev/stdin ) > & /dev/null
*#:: stdout stuff
*#: % ( mycmd > /dev/tty ) > & /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: % (mycmd >> out.txt) >& /dev/null
*# Write output to out.txt; store and watch errors ''Note: Compare the same task in ''sh''. ''
*#: % ( mycmd > out.txt ) | & tee err.txt
*#:: stderr stuff

* '''sh''' only
*# Discard errors, watch output
*#: $ mycmd 2> /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: $ mycmd 2> /dev/null >> out.txt
*# Write output to out.txt; store and watch errors ''Note: Compare the same task in ''tcsh''. ''
*#: $ mycmd 2>&1 > out.txt | tee err.txt
*#:: stderr stuff
*# Write messages to err.txt; write output to out.txt and copy output to terminal
*#: $ mycmd 2> err.txt | tee out.txt
*#:: stdout stuff
*# Assign a variable from stored.txt
*#: $ <stored.txt read var; mycmd $var
*#:: stderr stuff
*#:: stdout stuff
*# Assign first three lines of stored.txt to three different variables
*#: $ exec 3<&0; exec <stored.txt; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
*#:: stderr stuff
*#:: stdout stuff
*# Use all unique lines in stored.txt as an argument variable for ''mycmd'', appending to result.txt
*#: $ exec 3<&0; exec <stored.txt; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
*#:: stderr stuff
*: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 1> out.txt
*:: $ mycmd > out.txt
*::: stderr stuff
*: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 0< in.txt
*:: $ mycmd < in.txt
*::: stderr stuff
*::: stdout stuff
*: ''Note:'' To close a file descriptor, say <code>n<&-</code>
*:: $ mycmd >out.txt 1<&-;
*:: ''-sh: fails without messages''
*:: -bash: mycmd: write error: Bad file descriptor

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to another command. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 | command > errors.txt
because when you switched output to the messages file descriptor, it's done ''right away''. When you tried to switch errors to output there's nothing left to pipe. It's exactly as though you had said
samizdata# myprogram 1>/dev/stderr | command > errors.txt
''command'' receives no data, because it's all been sent to your screen. ''errors.txt'' will be empty. If you had tried it the other way, it's just as though you would have said:
samizdata# myprogram 2>/dev/stdout | command > errors.txt
In which case, errors and output are both piped to ''command'', which was not what you wanted.

This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, like this:
samizdata# myprogram 3>&2 2>&1 1>&3 | command > errors.txt

Let's step through what this does.
# ''fd 3'' points to <code>stderr</code>
# ''fd 2'' points to <code>stdout</code>
# ''fd 1'' points to ''fd 3'' (which is connected to <code>stderr</code>)
# ''|'' passes the open file descriptor connected to <code>stdout</code> (''fd 2'') making it <code>stdin</code> for ''command''
# ''>'' redirects the output of ''command'' to ''errors.txt''.

So, what you'll see on your screen is the output of ''myprogram'' on <code>stderr</code> (not passed to ''command''), followed by errors (if any) from ''command''. In short, what's been accomplished is, a swap of errors for output, from ''myprogram'' to ''command'', the output of which is redirected to ''errors.txt''. There is no reliable way to do the same thing in <code>tcsh</code>.

Also, see the [http://www.freebsd.org/cgi/man.cgi?query=mkfifo&apropos=0&sektion=1&manpath=FreeBSD+6.1-RELEASE&format=html man page] for [[mkfifo]], a utility for creating arbitrary file descriptor files.
[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Redirection

2006-06-28T03:24:49Z

Ninereasons: /* A little more about ''fd n'' in <code>sh</code> */ - perhaps even more clear

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Common operands include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipelines and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). When the next process inherits the open ''fd'' from the shell the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

* '''tcsh and sh'''
*# Write output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Append output to a file
*#: % mycmd >> out.txt
*#:: stderr stuff
*# Redirect the output of a remote command to ''local.txt''.
*#: % localcmd "remotecmd" > local.txt
*#:: stderr stuff
*# Same command as above, showing only the changes compared to ''local.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin'. These two commands are equivalent. ''
*#: % localcmd "remotecmd" | diff /dev/stdin local.txt
*#: % localcmd "remotecmd" | diff - local.txt
*#:: stderr stuff
*# Direct <code>stdout</code>+<code>stderr</code> to file
*#: % mycmd >& out.txt
*# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
*#: % <jumble.txt sort >sorted.txt
*#: % >sorted.txt sort <jumble.txt
*#: % sort < jumble.txt >sorted.txt
*#: % <jumble.txt>sorted.txt sort
*#:: stderr stuff
*# Sort unique lines of jumble.txt into sorted.txt
*#: % <jumble.txt sort | uniq >sorted.txt
*#:: stderr stuff
*# Sort ''HERE doc'' delimited by "lines"
*#: % <<lines sort 
*#::? a second line 
*#::? a first line 
*#::lines a first line a second line stderr stuff

* '''tcsh''' only
*# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
*#: % ( mycmd > /dev/stdin ) > & /dev/null
*#:: stdout stuff
*#: % ( mycmd > /dev/tty ) > & /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: % (mycmd >> out.txt) >& /dev/null
*# Write output to out.txt; store and watch errors ''Note: Compare the same task in ''sh''. ''
*#: % ( mycmd > out.txt ) | & tee err.txt
*#:: stderr stuff

* '''sh''' only
*# Discard errors, watch output
*#: $ mycmd 2> /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: $ mycmd 2> /dev/null >> out.txt
*# Write output to out.txt; store and watch errors ''Note: Compare the same task in ''tcsh''. ''
*#: $ mycmd 2>&1 > out.txt | tee err.txt
*#:: stderr stuff
*# Write messages to err.txt; write output to out.txt and copy output to terminal
*#: $ mycmd 2> err.txt | tee out.txt
*#:: stdout stuff
*# Assign a variable from stored.txt
*#: $ <stored.txt read var; mycmd $var
*#:: stderr stuff
*#:: stdout stuff
*# Assign first three lines of stored.txt to three different variables
*#: $ exec 3<&0; exec <stored.txt; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
*#:: stderr stuff
*#:: stdout stuff
*# Use all unique lines in stored.txt as an argument variable for ''mycmd'', appending to result.txt
*#: $ exec 3<&0; exec <stored.txt; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
*#:: stderr stuff
*: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 1> out.txt
*:: $ mycmd > out.txt
*::: stderr stuff
*: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 0< in.txt
*:: $ mycmd < in.txt
*::: stderr stuff
*::: stdout stuff
*: ''Note:'' To close a file descriptor, say <code>n<&-</code>
*:: $ mycmd >out.txt 1<&-;
*:: ''-sh: fails without messages''
*:: -bash: mycmd: write error: Bad file descriptor

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to another command. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 | command > errors.txt
because when you switched output to the messages file descriptor, it's done ''right away''. When you tried to switch errors to output there's nothing left to pipe. It's exactly as though you had said
samizdata# myprogram 1>/dev/stderr | command > errors.txt
''command'' receives no data, because it's all been sent to your screen. ''errors.txt'' will be empty. If you had tried it the other way, it's just as though you would have said:
samizdata# myprogram 2>/dev/stdout | command > errors.txt
In which case, errors and output are both piped to ''command'', which was not what you wanted.

This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, like this:
samizdata# myprogram 3>&2 2>&1 1>&3 | command > errors.txt

Let's step through what this does.
# ''fd 3'' points to <code>stderr</code>
# ''fd 2'' points to <code>stdout</code>
# ''fd 1'' points to ''fd 3'' (which is connected to <code>stderr</code>)
# ''|'' passes the open file descriptor connected to <code>stdout</code> (''fd 2'') making it <code>stdin</code> for ''command''
# ''>'' redirects the output of ''command'' to ''errors.txt''.

So, what you'll see on your screen is the output of ''myprogram'' on <code>stderr</code> (not passed to ''command''), followed by errors (if any) from ''command''. In short, what's been accomplished is, a swap of errors for output, from ''myprogram'' to ''command'', the output of which is redirected to ''errors.txt''. There is no reliable way to do the same thing in <code>tcsh</code>.

Also, see the [http://www.freebsd.org/cgi/man.cgi?query=mkfifo&apropos=0&sektion=1&manpath=FreeBSD+6.1-RELEASE&format=html man page] for [[mkfifo]], a utility for creating arbitrary file descriptor files.
[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Redirection

2006-06-28T03:17:13Z

Ninereasons: /* Redirection in <code>sh</code> compared to <code>tcsh</code> */

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Common operands include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipelines and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). When the next process inherits the open ''fd'' from the shell the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

* '''tcsh and sh'''
*# Write output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Append output to a file
*#: % mycmd >> out.txt
*#:: stderr stuff
*# Redirect the output of a remote command to ''local.txt''.
*#: % localcmd "remotecmd" > local.txt
*#:: stderr stuff
*# Same command as above, showing only the changes compared to ''local.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin'. These two commands are equivalent. ''
*#: % localcmd "remotecmd" | diff /dev/stdin local.txt
*#: % localcmd "remotecmd" | diff - local.txt
*#:: stderr stuff
*# Direct <code>stdout</code>+<code>stderr</code> to file
*#: % mycmd >& out.txt
*# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
*#: % <jumble.txt sort >sorted.txt
*#: % >sorted.txt sort <jumble.txt
*#: % sort < jumble.txt >sorted.txt
*#: % <jumble.txt>sorted.txt sort
*#:: stderr stuff
*# Sort unique lines of jumble.txt into sorted.txt
*#: % <jumble.txt sort | uniq >sorted.txt
*#:: stderr stuff
*# Sort ''HERE doc'' delimited by "lines"
*#: % <<lines sort 
*#::? a second line 
*#::? a first line 
*#::lines a first line a second line stderr stuff

* '''tcsh''' only
*# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
*#: % ( mycmd > /dev/stdin ) > & /dev/null
*#:: stdout stuff
*#: % ( mycmd > /dev/tty ) > & /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: % (mycmd >> out.txt) >& /dev/null
*# Write output to out.txt; store and watch errors ''Note: Compare the same task in ''sh''. ''
*#: % ( mycmd > out.txt ) | & tee err.txt
*#:: stderr stuff

* '''sh''' only
*# Discard errors, watch output
*#: $ mycmd 2> /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: $ mycmd 2> /dev/null >> out.txt
*# Write output to out.txt; store and watch errors ''Note: Compare the same task in ''tcsh''. ''
*#: $ mycmd 2>&1 > out.txt | tee err.txt
*#:: stderr stuff
*# Write messages to err.txt; write output to out.txt and copy output to terminal
*#: $ mycmd 2> err.txt | tee out.txt
*#:: stdout stuff
*# Assign a variable from stored.txt
*#: $ <stored.txt read var; mycmd $var
*#:: stderr stuff
*#:: stdout stuff
*# Assign first three lines of stored.txt to three different variables
*#: $ exec 3<&0; exec <stored.txt; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
*#:: stderr stuff
*#:: stdout stuff
*# Use all unique lines in stored.txt as an argument variable for ''mycmd'', appending to result.txt
*#: $ exec 3<&0; exec <stored.txt; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
*#:: stderr stuff
*: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 1> out.txt
*:: $ mycmd > out.txt
*::: stderr stuff
*: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 0< in.txt
*:: $ mycmd < in.txt
*::: stderr stuff
*::: stdout stuff
*: ''Note:'' To close a file descriptor, say <code>n<&-</code>
*:: $ mycmd >out.txt 1<&-;
*:: ''-sh: fails without messages''
*:: -bash: mycmd: write error: Bad file descriptor

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to another command. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 | command > errors.txt
because when you switched output to the messages file descriptor, it's done ''right away''. When you tried to switch errors to output there's nothing left to pipe. It's exactly as though you had said
samizdata# myprogram 1>&2 | command > errors.txt
''command'' receives no data, because it's all been sent to your screen. ''errors.txt'' will be empty. If you had tried it the other way, it's just as though you would have said:
samizdata# myprogram 2>&1 | command > errors.txt
In which case, errors and output are both piped to ''command'', which was not what you wanted.

This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, like this:
samizdata# myprogram 3>&2 2>&1 1>&3 | command > errors.txt

Let's step through what this does.
# ''fd 3'' points to <code>stderr</code>
# ''fd 2'' points to <code>stdout</code>
# ''fd 1'' points to ''fd 3'' (which is connected to <code>stderr</code>)
# ''|'' passes the open file descriptor connected to <code>stdout</code> (''fd 2'') making it <code>stdin</code> for ''command''
# ''>'' redirects the output of ''command'' to ''errors.txt''.

So, what you'll see on your screen is the output of ''myprogram'' on <code>stderr</code> (not passed to ''command''), followed by errors (if any) from ''command''. In short, what's been accomplished is, a swap of errors for output, from ''myprogram'' to ''command'', the output of which is redirected to ''errors.txt''. There is no reliable way to do the same thing in <code>tcsh</code>.

Also, see the [http://www.freebsd.org/cgi/man.cgi?query=mkfifo&apropos=0&sektion=1&manpath=FreeBSD+6.1-RELEASE&format=html man page] for [[mkfifo]], a utility for creating arbitrary file descriptor files.
[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Redirection

2006-06-28T03:15:21Z

Ninereasons: consistent terminology

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Common operands include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipelines and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). When the next process inherits the open ''fd'' from the shell the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

* '''tcsh and sh'''
*# Write output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Append output to a file
*#: % mycmd >> out.txt
*#:: stderr stuff
*# Redirect the output of a remote command to ''local.txt''.
*#: % localcmd "remotecmd" > local.txt
*#:: stderr stuff
*# Same command as above, showing only the changes compared to ''local.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin'. These two commands are equivalent. ''
*#: % localcmd "remotecmd" | diff /dev/stdin local.txt
*#: % localcmd "remotecmd" | diff - local.txt
*#:: stderr stuff
*# Direct <code>stdout</code>+<code>stderr</code> to file
*#: % mycmd >& out.txt
*# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
*#: % <jumble.txt sort >sorted.txt
*#: % >sorted.txt sort <jumble.txt
*#: % sort < jumble.txt >sorted.txt
*#: % <jumble.txt>sorted.txt sort
*#:: stderr stuff
*# Sort unique lines of jumble.txt into sorted.txt
*#: % <jumble.txt sort | uniq >sorted.txt
*#:: stderr stuff
*# Sort ''HERE doc'' delimited by "lines"
*#: % <<lines sort 
*#::? a second line 
*#::? a first line 
*#::lines a first line a second line stderr stuff

* '''tcsh''' only
*# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
*#: % ( mycmd > /dev/stdin ) > & /dev/null
*#:: stdout stuff
*#: % ( mycmd > /dev/tty ) > & /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: % (mycmd >> out.txt) >& /dev/null
*# Write output to out.txt; store and watch errors ''Note: Compare the same task in ''sh''. ''
*#: % ( mycmd > out.txt ) | & tee err.txt
*#:: stderr stuff

* '''sh''' only
*# Discard errors, watch output
*#: $ mycmd 2> /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: $ mycmd 2> /dev/null >> out.txt
*# Write output to out.txt; store and watch errors ''Note: Compare the same task in ''tcsh''. ''
*#: $ mycmd 2>&1 > out.txt | tee err.txt
*#:: stderr stuff
*# Write messages to err.txt; write output to out.txt and copy output to terminal
*#: $ mycmd 2> err.txt | tee out.txt
*#:: stdout stuff
*# Assign a variable from stored.txt
*#: $ <stored.txt read var; mycmd $var
*#:: stderr stuff
*#:: stdout stuff
*# Assign first three lines of stored.txt to three different variables
*#: $ exec 3<&0; exec <stored.txt; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
*#:: stderr stuff
*#:: stdout stuff
*# Use all unique lines in stored.txt as variable input, appending to result.txt
*#: $ exec 3<&0; exec <stored.txt; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
*#:: stderr stuff
*: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 1> out.txt
*:: $ mycmd > out.txt
*::: stderr stuff
*: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 0< in.txt
*:: $ mycmd < in.txt
*::: stderr stuff
*::: stdout stuff
*: ''Note:'' To close a file descriptor, say <code>n<&-</code>
*:: $ mycmd >out.txt 1<&-;
*:: ''-sh: fails without messages''
*:: -bash: mycmd: write error: Bad file descriptor

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to another command. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 | command > errors.txt
because when you switched output to the messages file descriptor, it's done ''right away''. When you tried to switch errors to output there's nothing left to pipe. It's exactly as though you had said
samizdata# myprogram 1>&2 | command > errors.txt
''command'' receives no data, because it's all been sent to your screen. ''errors.txt'' will be empty. If you had tried it the other way, it's just as though you would have said:
samizdata# myprogram 2>&1 | command > errors.txt
In which case, errors and output are both piped to ''command'', which was not what you wanted.

This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, like this:
samizdata# myprogram 3>&2 2>&1 1>&3 | command > errors.txt

Let's step through what this does.
# ''fd 3'' points to <code>stderr</code>
# ''fd 2'' points to <code>stdout</code>
# ''fd 1'' points to ''fd 3'' (which is connected to <code>stderr</code>)
# ''|'' passes the open file descriptor connected to <code>stdout</code> (''fd 2'') making it <code>stdin</code> for ''command''
# ''>'' redirects the output of ''command'' to ''errors.txt''.

So, what you'll see on your screen is the output of ''myprogram'' on <code>stderr</code> (not passed to ''command''), followed by errors (if any) from ''command''. In short, what's been accomplished is, a swap of errors for output, from ''myprogram'' to ''command'', the output of which is redirected to ''errors.txt''. There is no reliable way to do the same thing in <code>tcsh</code>.

Also, see the [http://www.freebsd.org/cgi/man.cgi?query=mkfifo&apropos=0&sektion=1&manpath=FreeBSD+6.1-RELEASE&format=html man page] for [[mkfifo]], a utility for creating arbitrary file descriptor files.
[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Redirection

2006-06-28T02:41:52Z

Ninereasons: /* A little more about ''fd n'' in <code>sh</code> */ - clarify stream swapping example

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Common operands include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipelines and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). When the next process inherits the open ''fd'' from the shell the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

* '''tcsh and sh'''
*# Write output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Append output to a file
*#: % mycmd >> out.txt
*#:: stderr stuff
*# Redirect the output of a remote command to ''local.txt''.
*#: % localcmd "remotecmd" > local.txt
*#:: stderr stuff
*# Same command as above, showing only the changes compared to ''local.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin'. These two commands are equivalent. ''
*#: % localcmd "remotecmd" | diff /dev/stdin local.txt
*#: % localcmd "remotecmd" | diff - local.txt
*#:: stderr stuff
*# Direct <code>stdout</code>+<code>stderr</code> to file
*#: % localcmd >& out.txt
*# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
*#: % <jumble.txt sort >sorted.txt
*#: % >sorted.txt sort <jumble.txt
*#: % sort < jumble.txt >sorted.txt
*#: % <jumble.txt>sorted.txt sort
*#:: stderr stuff
*# Sort unique lines of jumble.txt into sorted.txt
*#: % <jumble.txt sort | uniq >sorted.txt
*#:: stderr stuff
*# Sort ''HERE doc'' delimited by "lines"
*#: % <<lines sort 
*#::? a second line 
*#::? a first line 
*#::lines a first line a second line stderr stuff

* '''tcsh''' only
*# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
*#: % ( mycmd > /dev/stdin ) > & /dev/null
*#:: stdout stuff
*#: % ( mycmd > /dev/tty ) > & /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: % (mycmd >> out.txt) >& /dev/null
*# Write output to out.txt; store and watch errors ''Note: Compare the same task in ''sh''. ''
*#: % ( mycmd > out.txt ) | & tee err.txt
*#:: stderr stuff

* '''sh''' only
*# Discard errors, watch output
*#: $ mycmd 2> /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: $ mycmd 2> /dev/null >> out.txt
*# Write output to out.txt; store and watch errors ''Note: Compare the same task in ''tcsh''. ''
*#: $ mycmd 2>&1 > out.txt | tee err.txt
*#:: stderr stuff
*# Write messages to err.txt; write output to out.txt and copy output to terminal
*#: $ mycmd 2> err.txt | tee out.txt
*#:: stdout stuff
*# Assign a variable from stored.txt
*#: $ <stored.txt read var; mycmd $var
*#:: stderr stuff
*#:: stdout stuff
*# Assign first three lines of stored.txt to three different variables
*#: $ exec 3<&0; exec <stored.txt; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
*#:: stderr stuff
*#:: stdout stuff
*# Use all unique lines in stored.txt as variable input, appending to result.txt
*#: $ exec 3<&0; exec <stored.txt; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
*#:: stderr stuff
*: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 1> out.txt
*:: $ mycmd > out.txt
*::: stderr stuff
*: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 0< in.txt
*:: $ mycmd < in.txt
*::: stderr stuff
*::: stdout stuff
*: ''Note:'' To close a file descriptor, say <code>n<&-</code>
*:: $ mycmd >out.txt 1<&-;
*:: ''-sh: fails without messages''
*:: -bash: mycmd: write error: Bad file descriptor

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to another command. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 | command > errors.txt
because when you switched output to the messages file descriptor, it's done ''right away''. When you tried to switch errors to output there's nothing left to pipe. It's exactly as though you had said
samizdata# myprogram 1>&2 | command > errors.txt
''command'' receives no data, because it's all been sent to your screen. ''errors.txt'' will be empty. If you had tried it the other way, it's just as though you would have said:
samizdata# myprogram 2>&1 | command > errors.txt
In which case, errors and output are both piped to ''command'', which was not what you wanted.

This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, like this:
samizdata# myprogram 3>&2 2>&1 1>&3 | command > errors.txt

Let's step through what this does.
# ''fd 3'' points to <code>stderr</code>
# ''fd 2'' points to <code>stdout</code>
# ''fd 1'' points to ''fd 3'' (which is connected to <code>stderr</code>)
# ''|'' passes the open file descriptor connected to <code>stdout</code> (''fd 2'') making it <code>stdin</code> for ''command''
# ''>'' redirects the output of ''command'' to ''errors.txt''.

So, what you'll see on your screen is the output of ''myprogram'' on <code>stderr</code> (not passed to ''command''), followed by errors (if any) from ''command''. In short, what's been accomplished is, a swap of errors for output, from ''myprogram'' to ''command'', the output of which is redirected to ''errors.txt''. There is no reliable way to do the same thing in <code>tcsh</code>.

Also, see the [http://www.freebsd.org/cgi/man.cgi?query=mkfifo&apropos=0&sektion=1&manpath=FreeBSD+6.1-RELEASE&format=html man page] for [[mkfifo]], a utility for creating arbitrary file descriptor files.
[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Redirection

2006-06-28T01:40:40Z

Ninereasons: filename incorrect

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Common operands include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipelines and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). When the next process inherits the open ''fd'' from the shell the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

* '''tcsh and sh'''
*# Write output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Append output to a file
*#: % mycmd >> out.txt
*#:: stderr stuff
*# Redirect the output of a remote command to ''local.txt''.
*#: % localcmd "remotecmd" > local.txt
*#:: stderr stuff
*# Same command as above, showing only the changes compared to ''local.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin'. These two commands are equivalent. ''
*#: % localcmd "remotecmd" | diff /dev/stdin local.txt
*#: % localcmd "remotecmd" | diff - local.txt
*#:: stderr stuff
*# Direct <code>stdout</code>+<code>stderr</code> to file
*#: % localcmd >& out.txt
*# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
*#: % <jumble.txt sort >sorted.txt
*#: % >sorted.txt sort <jumble.txt
*#: % sort < jumble.txt >sorted.txt
*#: % <jumble.txt>sorted.txt sort
*#:: stderr stuff
*# Sort unique lines of jumble.txt into sorted.txt
*#: % <jumble.txt sort | uniq >sorted.txt
*#:: stderr stuff
*# Sort ''HERE doc'' delimited by "lines"
*#: % <<lines sort 
*#::? a second line 
*#::? a first line 
*#::lines a first line a second line stderr stuff

* '''tcsh''' only
*# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
*#: % ( mycmd > /dev/stdin ) > & /dev/null
*#:: stdout stuff
*#: % ( mycmd > /dev/tty ) > & /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: % (mycmd >> out.txt) >& /dev/null
*# Write output to out.txt; store and watch errors ''Note: Compare the same task in ''sh''. ''
*#: % ( mycmd > out.txt ) | & tee err.txt
*#:: stderr stuff

* '''sh''' only
*# Discard errors, watch output
*#: $ mycmd 2> /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: $ mycmd 2> /dev/null >> out.txt
*# Write output to out.txt; store and watch errors ''Note: Compare the same task in ''tcsh''. ''
*#: $ mycmd 2>&1 > out.txt | tee err.txt
*#:: stderr stuff
*# Write messages to err.txt; write output to out.txt and copy output to terminal
*#: $ mycmd 2> err.txt | tee out.txt
*#:: stdout stuff
*# Assign a variable from stored.txt
*#: $ <stored.txt read var; mycmd $var
*#:: stderr stuff
*#:: stdout stuff
*# Assign first three lines of stored.txt to three different variables
*#: $ exec 3<&0; exec <stored.txt; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
*#:: stderr stuff
*#:: stdout stuff
*# Use all unique lines in stored.txt as variable input, appending to result.txt
*#: $ exec 3<&0; exec <stored.txt; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
*#:: stderr stuff
*: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 1> out.txt
*:: $ mycmd > out.txt
*::: stderr stuff
*: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 0< in.txt
*:: $ mycmd < in.txt
*::: stderr stuff
*::: stdout stuff
*: ''Note:'' To close a file descriptor, say <code>n<&-</code>
*:: $ mycmd >out.txt 1<&-;
*:: ''-sh: fails without messages''
*:: -bash: mycmd: write error: Bad file descriptor

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to a file. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 > errors.txt
because when you do the first switch, it's done ''right away'' and when the second >& comes around, it's getting the switched data. This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, such as:
samizdata# myprogram 3>&2 2>&1 1>&3 | command
will make the output of ''myprogram'' do this: 3 point to the same place as 2, 2 point to 1, and finally, 1 point to 3 and then pipe all of it to ''command''

Also, see the [http://www.freebsd.org/cgi/man.cgi?query=mkfifo&apropos=0&sektion=1&manpath=FreeBSD+6.1-RELEASE&format=html man page] for [[mkfifo]], a utility for creating arbitrary file descriptor files.
[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Redirection

2006-06-28T01:35:04Z

Ninereasons: better example

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Common operands include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipelines and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). When the next process inherits the open ''fd'' from the shell the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

* '''tcsh and sh'''
*# Write output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Append output to a file
*#: % mycmd >> out.txt
*#:: stderr stuff
*# Redirect the output of a remote command to ''local.txt''.
*#: % localcmd "remotecmd" > local.txt
*#:: stderr stuff
*# Same command as above, showing only the changes compared to ''local.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin'. These two commands are equivalent. ''
*#: % localcmd "remotecmd" | diff /dev/stdin local.txt
*#: % localcmd "remotecmd" | diff - local.txt
*#:: stderr stuff
*# Direct <code>stdout</code>+<code>stderr</code> to file
*#: % localcmd >& out.txt
*# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
*#: % <jumble.txt sort >sorted.txt
*#: % >sorted.txt sort <jumble.txt
*#: % sort < jumble.txt >sorted.txt
*#: % <jumble.txt>sorted.txt sort
*#:: stderr stuff
*# Sort unique lines of jumble.txt into sorted.txt
*#: % <jumble.txt sort | uniq >sorted.txt
*#:: stderr stuff
*# Sort ''HERE doc'' delimited by "lines"
*#: % <<lines sort 
*#::? a second line 
*#::? a first line 
*#::lines a first line a second line stderr stuff

* '''tcsh''' only
*# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
*#: % ( mycmd > /dev/stdin ) > & /dev/null
*#:: stdout stuff
*#: % ( mycmd > /dev/tty ) > & /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: % (mycmd >> out.txt) >& /dev/null
*# Write output to out.txt; store and watch errors ''Note: Compare the same task in ''sh''. ''
*#: % ( mycmd > out.txt ) | & tee err.txt
*#:: stderr stuff

* '''sh''' only
*# Discard errors, watch output
*#: $ mycmd 2> /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: $ mycmd 2> /dev/null >> out.txt
*# Write output to out.txt; store and watch errors ''Note: Compare the same task in ''tcsh''. ''
*#: $ mycmd 2>&1 > out.txt | tee err.txt
*#:: stderr stuff
*# Write messages to err.txt; write output to out.txt and copy output to terminal
*#: $ mycmd 2> err.txt | tee out.txt
*#:: stdout stuff
*# Assign a variable from stored.txt
*#: $ <stored.txt read var; mycmd $var
*#:: stderr stuff
*#:: stdout stuff
*# Assign first three lines of stored.txt to three different variables
*#: $ exec 3<&0; exec <test; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
*#:: stderr stuff
*#:: stdout stuff
*# Use all unique lines in stored.txt as variable input, appending to result.txt
*#: $ exec 3<&0; exec <test; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
*#:: stderr stuff
*: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 1> out.txt
*:: $ mycmd > out.txt
*::: stderr stuff
*: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 0< in.txt
*:: $ mycmd < in.txt
*::: stderr stuff
*::: stdout stuff
*: ''Note:'' To close a file descriptor, say <code>n<&-</code>
*:: $ mycmd >out.txt 1<&-;
*:: ''-sh: fails without messages''
*:: -bash: mycmd: write error: Bad file descriptor

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to a file. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 > errors.txt
because when you do the first switch, it's done ''right away'' and when the second >& comes around, it's getting the switched data. This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, such as:
samizdata# myprogram 3>&2 2>&1 1>&3 | command
will make the output of ''myprogram'' do this: 3 point to the same place as 2, 2 point to 1, and finally, 1 point to 3 and then pipe all of it to ''command''

Also, see the [http://www.freebsd.org/cgi/man.cgi?query=mkfifo&apropos=0&sektion=1&manpath=FreeBSD+6.1-RELEASE&format=html man page] for [[mkfifo]], a utility for creating arbitrary file descriptor files.
[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Redirection

2006-06-28T00:59:29Z

Ninereasons: /* Redirection in <code>sh</code> compared to <code>tcsh</code> */

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Common operands include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipelines and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). When the next process inherits the open ''fd'' from the shell the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

* '''tcsh and sh'''
*# Write output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Append output to a file
*#: % mycmd >> out.txt
*#:: stderr stuff
*# Redirect the output of a remote command to ''local.txt''.
*#: % localcmd "remotecmd" > local.txt
*#:: stderr stuff
*# Same command as above, showing only the changes compared to ''local.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin'. These two commands are equivalent. ''
*#: % localcmd "remotecmd" | diff /dev/stdin local.txt
*#: % localcmd "remotecmd" | diff - local.txt
*#:: stderr stuff
*# Direct <code>stdout</code>+<code>stderr</code> to file
*#: % localcmd >& out.txt
*# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
*#: % <jumble.txt sort >sorted.txt
*#: % >sorted.txt sort <jumble.txt
*#: % sort < jumble.txt >sorted.txt
*#: % <jumble.txt>sorted.txt sort
*#:: stderr stuff
*# Sort unique lines of jumble.txt into sorted.txt
*#: % <jumble.txt sort | uniq >sorted.txt
*#:: stderr stuff
*# Sort ''HERE doc'' delimited by "lines"
*#: % <<lines sort 
*#::? a second line 
*#::? a first line 
*#::lines a first line a second line stderr stuff

* '''tcsh''' only
*# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
*#: % ( mycmd > /dev/stdin ) > & /dev/null
*#:: stdout stuff
*#: % ( mycmd > /dev/tty ) > & /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: % (mycmd >> out.txt) >& /dev/null
*# Write output to out.txt; store and watch errors ''Note: this happens to be easier in ''tcsh'' - a rare event. Compare the same task in ''sh''. ''
*#: % ( mycmd > out.txt ) | & tee err.txt
*#:: stderr stuff

* '''sh''' only
*# Discard errors, watch output
*#: $ mycmd 2> /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: $ mycmd 2> /dev/null >> out.txt
*# Write output to out.txt; store and watch errors ''Note: this happens to be harder in ''sh'' - a rare event. Compare the same task in ''tcsh''. ''
*#: $ exec 3>&1 ; mycmd 2>&1 >&3 1>out.txt | tee err.txt ; exec 1<&3 3<&-;
*#:: stderr stuff
*# Write messages to err.txt; write output to out.txt and copy output to terminal
*#: $ mycmd 2> err.txt | tee out.txt
*#:: stdout stuff
*# Assign a variable from stored.txt
*#: $ <stored.txt read var; mycmd $var
*#:: stderr stuff
*#:: stdout stuff
*# Assign first three lines of stored.txt to three different variables
*#: $ exec 3<&0; exec <test; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
*#:: stderr stuff
*#:: stdout stuff
*# Use all unique lines in stored.txt as variable input, appending to result.txt
*#: $ exec 3<&0; exec <test; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
*#:: stderr stuff
*: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 1> out.txt
*:: $ mycmd > out.txt
*::: stderr stuff
*: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 0< in.txt
*:: $ mycmd < in.txt
*::: stderr stuff
*::: stdout stuff
*: ''Note:'' To close a file descriptor, say <code>n<&-</code>
*:: $ mycmd >out.txt 1<&-;
*:: ''-sh: fails without messages''
*:: -bash: mycmd: write error: Bad file descriptor

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to a file. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 > errors.txt
because when you do the first switch, it's done ''right away'' and when the second >& comes around, it's getting the switched data. This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, such as:
samizdata# myprogram 3>&2 2>&1 1>&3 | command
will make the output of ''myprogram'' do this: 3 point to the same place as 2, 2 point to 1, and finally, 1 point to 3 and then pipe all of it to ''command''

Also, see the [http://www.freebsd.org/cgi/man.cgi?query=mkfifo&apropos=0&sektion=1&manpath=FreeBSD+6.1-RELEASE&format=html man page] for [[mkfifo]], a utility for creating arbitrary file descriptor files.
[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Redirection

2006-06-28T00:54:28Z

Ninereasons: correct what is sent to the screen

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Common operands include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipelines and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). When the next process inherits the open ''fd'' from the shell the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

* '''tcsh and sh'''
*# Write output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Append output to a file
*#: % mycmd >> out.txt
*#:: stderr stuff
*# Redirect the output of a remote command to ''local.txt''.
*#: % localcmd "remotecmd" > local.txt
*#:: stderr stuff
*# Same command as above, showing only the changes compared to ''local.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin'. These two commands are equivalent. ''
*#: % localcmd "remotecmd" | diff /dev/stdin local.txt
*#: % localcmd "remotecmd" | diff - local.txt
*#:: stderr stuff
*# Direct <code>stdout</code>+<code>stderr</code> to file
*#: % localcmd >& out.txt
*# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
*#: % <jumble.txt sort >sorted.txt
*#: % >sorted.txt sort <jumble.txt
*#: % sort < jumble.txt >sorted.txt
*#: % <jumble.txt>sorted.txt sort
*#:: stderr stuff
*# Sort unique lines of jumble.txt into sorted.txt
*#: % <jumble.txt sort | uniq >sorted.txt
*#:: stderr stuff
*# Sort ''HERE doc'' delimited by "lines"
*#: % <<lines sort 
*#::? a second line 
*#::? a first line 
*#::lines a first line a second line stderr stuff

* '''tcsh''' only
*# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
*#: % ( mycmd > /dev/stdin ) > & /dev/null
*#:: stdout stuff
*#: % ( mycmd > /dev/tty ) > & /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: % (mycmd >> out.txt) >& /dev/null
*# Write output to out.txt; store and watch errors ''Note: this happens to be easier in ''tcsh'' - a rare event. Compare the same task in ''sh''. ''
*#: % ( mycmd > out.txt ) | & tee err.txt
*#:: stderr stuff

* '''sh''' only
*# Discard errors, watch output
*#: $ mycmd 2> /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: $ mycmd 2> /dev/null >> out.txt
*# Write output to out.txt; store and watch errors ''Note: this happens to be harder in ''sh'' - a rare event. Compare the same task in ''tcsh''. ''
*#: $ exec 3>&1 ; mycmd 2>&1 >&3 1>out.txt | tee err.txt ; exec 1<&3 3<&-;
*#:: stderr stuff
*# Write messages to err.txt; write output to out.txt and copy output to terminal
*#: $ mycmd 2> err.txt | tee out.txt
*#:: stdout stuff
*# Assign a variable from stored.txt
*#: $ <stored.txt read var; mycmd $var
*#:: stdout stuff
*# Assign first three lines of stored.txt to three different variables
*#: $ exec 3<&0; exec <test; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
*#:: stderr stuff
*#:: stdout stuff
*# Use all unique lines in stored.txt as variable input, appending to result.txt
*#: $ exec 3<&0; exec <test; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
*#:: stderr stuff
*: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 1> out.txt
*:: $ mycmd > out.txt
*::: stderr stuff
*: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 0< in.txt
*:: $ mycmd < in.txt
*::: stderr stuff
*::: stdout stuff
*: ''Note:'' To close a file descriptor, say <code>n<&-</code>
*:: $ mycmd >out.txt 1<&-;
*:: ''-sh: fails without messages''
*:: -bash: mycmd: write error: Bad file descriptor

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to a file. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 > errors.txt
because when you do the first switch, it's done ''right away'' and when the second >& comes around, it's getting the switched data. This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, such as:
samizdata# myprogram 3>&2 2>&1 1>&3 | command
will make the output of ''myprogram'' do this: 3 point to the same place as 2, 2 point to 1, and finally, 1 point to 3 and then pipe all of it to ''command''

Also, see the [http://www.freebsd.org/cgi/man.cgi?query=mkfifo&apropos=0&sektion=1&manpath=FreeBSD+6.1-RELEASE&format=html man page] for [[mkfifo]], a utility for creating arbitrary file descriptor files.
[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Redirection

2006-06-28T00:52:04Z

Ninereasons: another equivalent

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Common operands include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipelines and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). When the next process inherits the open ''fd'' from the shell the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

* '''tcsh and sh'''
*# Write output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Append output to a file
*#: % mycmd >> out.txt
*#:: stderr stuff
*# Redirect the output of a remote command to ''local.txt''.
*#: % localcmd "remotecmd" > local.txt
*#:: stderr stuff
*# Same command as above, showing only the changes compared to ''local.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin'. These two commands are equivalent. ''
*#: % localcmd "remotecmd" | diff /dev/stdin local.txt
*#: % localcmd "remotecmd" | diff - local.txt
*#:: stderr stuff
*# Direct <code>stdout</code>+<code>stderr</code> to file
*#: % localcmd >& out.txt
*# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
*#: % <jumble.txt sort >sorted.txt
*#: % >sorted.txt sort <jumble.txt
*#: % sort < jumble.txt >sorted.txt
*#: % <jumble.txt>sorted.txt sort
*#:: stdout stuff
*# Sort unique lines of jumble.txt into sorted.txt
*#: % <jumble.txt sort | uniq >sorted.txt
*#:: stderr stuff
*# Sort ''HERE doc'' delimited by "lines"
*#: % <<lines sort 
*#::? a second line 
*#::? a first line 
*#::lines a first line a second line stderr stuff

* '''tcsh''' only
*# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
*#: % ( mycmd > /dev/stdin ) > & /dev/null
*#:: stdout stuff
*#: % ( mycmd > /dev/tty ) > & /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: % (mycmd >> out.txt) >& /dev/null
*# Write output to out.txt; store and watch errors ''Note: this happens to be easier in ''tcsh'' - a rare event. Compare the same task in ''sh''. ''
*#: % ( mycmd > out.txt ) | & tee err.txt
*#:: stderr stuff

* '''sh''' only
*# Discard errors, watch output
*#: $ mycmd 2> /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: $ mycmd 2> /dev/null >> out.txt
*# Write output to out.txt; store and watch errors ''Note: this happens to be harder in ''sh'' - a rare event. Compare the same task in ''tcsh''. ''
*#: $ exec 3>&1 ; mycmd 2>&1 >&3 1>out.txt | tee err.txt ; exec 1<&3 3<&-;
*#:: stderr stuff
*# Write messages to err.txt; write output to out.txt and copy output to terminal
*#: $ mycmd 2> err.txt | tee out.txt
*#:: stdout stuff
*# Assign a variable from stored.txt
*#: $ <stored.txt read var; mycmd $var
*#:: stdout stuff
*# Assign first three lines of stored.txt to three different variables
*#: $ exec 3<&0; exec <test; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
*#:: stderr stuff
*#:: stdout stuff
*# Use all unique lines in stored.txt as variable input, appending to result.txt
*#: $ exec 3<&0; exec <test; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
*#:: stderr stuff
*: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 1> out.txt
*:: $ mycmd > out.txt
*::: stderr stuff
*: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 0< in.txt
*:: $ mycmd < in.txt
*::: stderr stuff
*::: stdout stuff
*: ''Note:'' To close a file descriptor, say <code>n<&-</code>
*:: $ mycmd >out.txt 1<&-;
*:: ''-sh: fails without messages''
*:: -bash: mycmd: write error: Bad file descriptor

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to a file. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 > errors.txt
because when you do the first switch, it's done ''right away'' and when the second >& comes around, it's getting the switched data. This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, such as:
samizdata# myprogram 3>&2 2>&1 1>&3 | command
will make the output of ''myprogram'' do this: 3 point to the same place as 2, 2 point to 1, and finally, 1 point to 3 and then pipe all of it to ''command''

Also, see the [http://www.freebsd.org/cgi/man.cgi?query=mkfifo&apropos=0&sektion=1&manpath=FreeBSD+6.1-RELEASE&format=html man page] for [[mkfifo]], a utility for creating arbitrary file descriptor files.
[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

PHP

2006-06-27T22:50:18Z

Ninereasons: /* Installing PHP */

PHP is the recursive acronym for ''PHP Hypertext Preprocessor''. It is an interpreted script language commonly used for dynamic pages generation on webservers. Then it is generaly installed side by side with the [[Apache]] web server, and [[MySQL]] or [[PostgreSQL]] as database management system.

==Installing PHP==

PHP is available from the port-tree ...
# cd /usr/ports/lang/php5
... and the packages ...
# pkg_add -r php5

==Using PHP==
There are two ways to use PHP :
* With apache: configure apache to use php to process .php files requested by the user.
* As a command interpreter: php can process scripts, or instructions typed at the commandline.

===PHP for dynamic websites===

All the steps of Apache configuration are explained after PHP installation. You have to add these two lines in httpd.conf:
AddType application/x-httpd-php .php
AddType application/x-httpd-php-source .phps

You will probably also want to add ''index.php'' as a possible directory index:
DirectoryIndex index.html index.php

You have to restart Apache to have your changes effective. For example, if you use Apache 2:
/usr/local/etc/rc.d/apache2 restart

Now, you can try your installation by creating a file test.php:
<?
phpinfo();
?>

Request test.php using your web browser, and if all is well it will show you your PHP configuration.

===Scripts in PHP===

If you have compiled PHP with the commandline interface, you can write scripts in PHP that can be used from the [[shell]]. This is very useful for ''quick and dirty'' hacks; especially if you want your output to be HTML. Put the PHP code in a file like this :
<pre>
#!/usr/local/bin/php
# phpscript
<html> <title>Defined Constants</title> <body>

<h1><? echo "Hello from PHP " .phpversion() ?></h1>
<pre><? print_r(get_defined_constants()) ?></pre>

</body></html>
</pre>

Make the file executable:
'''%''' chmod +x phpscript

Then, you can run it from the commandline.

'''%''' ./phpscript | lynx -stdin
In this case, we've piped to <code>lynx</code>, so that we can browse the nicely formatted result.
<pre>
Defined Constants (p1 of 154)

Hello from PHP 5.1.4

Array
(
[E_ERROR] => 1
[E_WARNING] => 2
[E_PARSE] => 4
[E_NOTICE] => 8
-- press space for more, use arrow keys to move, '?' for help, 'q' to quit.
</pre>

Or, we could have executed the command directly from the commandline:
'''%''' php -r 'echo "Hello from PHP ". phpversion() . "\n"; print_r(get_defined_constants());'
Hello from PHP 5.1.4
Array
(
...

==PHP Website==
[http://www.php.net http://www.php.net]

[[Category: Ports and Packages]]
[[Category: Configuring FreeBSD]]

Redirection

2006-06-27T22:23:42Z

Ninereasons: /* Shell pipes and redirection */ - clarify a little

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Common operands include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipelines and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). When the next process inherits the open ''fd'' from the shell the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

* '''tcsh and sh'''
*# Write output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Append output to a file
*#: % mycmd >> out.txt
*#:: stderr stuff
*# Redirect the output of a remote command to ''local.txt''.
*#: % localcmd "remotecmd" > local.txt
*#:: stderr stuff
*# Same command as above, showing only the changes compared to ''local.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin'. These two commands are equivalent. ''
*#: % localcmd "remotecmd" | diff /dev/stdin local.txt
*#: % localcmd "remotecmd" | diff - local.txt
*#:: stderr stuff
*# Direct <code>stdout</code>+<code>stderr</code> to file
*#: % localcmd >& out.txt
*# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
*#: % <jumble.txt sort >sorted.txt
*#: % >sorted.txt sort <jumble.txt
*#: % sort < jumble.txt >sorted.txt
*#:: stdout stuff
*# Sort unique lines of jumble.txt into sorted.txt
*#: % <jumble.txt sort | uniq >sorted.txt
*#:: stderr stuff
*# Sort ''HERE doc'' delimited by "lines"
*#: % <<lines sort 
*#::? a second line 
*#::? a first line 
*#::lines a first line a second line stderr stuff

* '''tcsh''' only
*# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
*#: % ( mycmd > /dev/stdin ) > & /dev/null
*#:: stdout stuff
*#: % ( mycmd > /dev/tty ) > & /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: % (mycmd >> out.txt) >& /dev/null
*# Write output to out.txt; store and watch errors ''Note: this happens to be easier in ''tcsh'' - a rare event. Compare the same task in ''sh''. ''
*#: % ( mycmd > out.txt ) | & tee err.txt
*#:: stderr stuff

* '''sh''' only
*# Discard errors, watch output
*#: $ mycmd 2> /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: $ mycmd 2> /dev/null >> out.txt
*# Write output to out.txt; store and watch errors ''Note: this happens to be harder in ''sh'' - a rare event. Compare the same task in ''tcsh''. ''
*#: $ exec 3>&1 ; mycmd 2>&1 >&3 1>out.txt | tee err.txt ; exec 1<&3 3<&-;
*#:: stderr stuff
*# Write messages to err.txt; write output to out.txt and copy output to terminal
*#: $ mycmd 2> err.txt | tee out.txt
*#:: stdout stuff
*# Assign a variable from stored.txt
*#: $ <stored.txt read var; mycmd $var
*#:: stdout stuff
*# Assign first three lines of stored.txt to three different variables
*#: $ exec 3<&0; exec <test; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
*#:: stderr stuff
*#:: stdout stuff
*# Use all unique lines in stored.txt as variable input, appending to result.txt
*#: $ exec 3<&0; exec <test; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
*#:: stderr stuff
*: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 1> out.txt
*:: $ mycmd > out.txt
*::: stderr stuff
*: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 0< in.txt
*:: $ mycmd < in.txt
*::: stderr stuff
*::: stdout stuff
*: ''Note:'' To close a file descriptor, say <code>n<&-</code>
*:: $ mycmd >out.txt 1<&-;
*:: ''-sh: fails without messages''
*:: -bash: mycmd: write error: Bad file descriptor

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to a file. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 > errors.txt
because when you do the first switch, it's done ''right away'' and when the second >& comes around, it's getting the switched data. This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, such as:
samizdata# myprogram 3>&2 2>&1 1>&3 | command
will make the output of ''myprogram'' do this: 3 point to the same place as 2, 2 point to 1, and finally, 1 point to 3 and then pipe all of it to ''command''

Also, see the [http://www.freebsd.org/cgi/man.cgi?query=mkfifo&apropos=0&sektion=1&manpath=FreeBSD+6.1-RELEASE&format=html man page] for [[mkfifo]], a utility for creating arbitrary file descriptor files.
[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Redirection

2006-06-27T21:48:31Z

Ninereasons: /* A little more about ''fd n'' in <code>sh</code> */

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Common operands include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipes and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). The newly assigned process can reassign the open ''fd'' in turn, so that the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

* '''tcsh and sh'''
*# Write output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Append output to a file
*#: % mycmd >> out.txt
*#:: stderr stuff
*# Redirect the output of a remote command to ''local.txt''.
*#: % localcmd "remotecmd" > local.txt
*#:: stderr stuff
*# Same command as above, showing only the changes compared to ''local.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin'. These two commands are equivalent. ''
*#: % localcmd "remotecmd" | diff /dev/stdin local.txt
*#: % localcmd "remotecmd" | diff - local.txt
*#:: stderr stuff
*# Direct <code>stdout</code>+<code>stderr</code> to file
*#: % localcmd >& out.txt
*# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
*#: % <jumble.txt sort >sorted.txt
*#: % >sorted.txt sort <jumble.txt
*#: % sort < jumble.txt >sorted.txt
*#:: stdout stuff
*# Sort unique lines of jumble.txt into sorted.txt
*#: % <jumble.txt sort | uniq >sorted.txt
*#:: stderr stuff
*# Sort ''HERE doc'' delimited by "lines"
*#: % <<lines sort 
*#::? a second line 
*#::? a first line 
*#::lines a first line a second line stderr stuff

* '''tcsh''' only
*# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
*#: % ( mycmd > /dev/stdin ) > & /dev/null
*#:: stdout stuff
*#: % ( mycmd > /dev/tty ) > & /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: % (mycmd >> out.txt) >& /dev/null
*# Write output to out.txt; store and watch errors ''Note: this happens to be easier in ''tcsh'' - a rare event. Compare the same task in ''sh''. ''
*#: % ( mycmd > out.txt ) | & tee err.txt
*#:: stderr stuff

* '''sh''' only
*# Discard errors, watch output
*#: $ mycmd 2> /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: $ mycmd 2> /dev/null >> out.txt
*# Write output to out.txt; store and watch errors ''Note: this happens to be harder in ''sh'' - a rare event. Compare the same task in ''tcsh''. ''
*#: $ exec 3>&1 ; mycmd 2>&1 >&3 1>out.txt | tee err.txt ; exec 1<&3 3<&-;
*#:: stderr stuff
*# Write messages to err.txt; write output to out.txt and copy output to terminal
*#: $ mycmd 2> err.txt | tee out.txt
*#:: stdout stuff
*# Assign a variable from stored.txt
*#: $ <stored.txt read var; mycmd $var
*#:: stdout stuff
*# Assign first three lines of stored.txt to three different variables
*#: $ exec 3<&0; exec <test; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
*#:: stderr stuff
*#:: stdout stuff
*# Use all unique lines in stored.txt as variable input, appending to result.txt
*#: $ exec 3<&0; exec <test; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
*#:: stderr stuff
*: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 1> out.txt
*:: $ mycmd > out.txt
*::: stderr stuff
*: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 0< in.txt
*:: $ mycmd < in.txt
*::: stderr stuff
*::: stdout stuff
*: ''Note:'' To close a file descriptor, say <code>n<&-</code>
*:: $ mycmd >out.txt 1<&-;
*:: ''-sh: fails without messages''
*:: -bash: mycmd: write error: Bad file descriptor

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to a file. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 > errors.txt
because when you do the first switch, it's done ''right away'' and when the second >& comes around, it's getting the switched data. This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, such as:
samizdata# myprogram 3>&2 2>&1 1>&3 | command
will make the output of ''myprogram'' do this: 3 point to the same place as 2, 2 point to 1, and finally, 1 point to 3 and then pipe all of it to ''command''

Also, see the [http://www.freebsd.org/cgi/man.cgi?query=mkfifo&apropos=0&sektion=1&manpath=FreeBSD+6.1-RELEASE&format=html man page] for [[mkfifo]], a utility for creating arbitrary file descriptor files.
[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Redirection

2006-06-27T21:10:55Z

Ninereasons: closed file descriptors

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Common operands include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipes and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). The newly assigned process can reassign the open ''fd'' in turn, so that the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

* '''tcsh and sh'''
*# Write output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Append output to a file
*#: % mycmd >> out.txt
*#:: stderr stuff
*# Redirect the output of a remote command to ''local.txt''.
*#: % localcmd "remotecmd" > local.txt
*#:: stderr stuff
*# Same command as above, showing only the changes compared to ''local.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin'. These two commands are equivalent. ''
*#: % localcmd "remotecmd" | diff /dev/stdin local.txt
*#: % localcmd "remotecmd" | diff - local.txt
*#:: stderr stuff
*# Direct <code>stdout</code>+<code>stderr</code> to file
*#: % localcmd >& out.txt
*# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
*#: % <jumble.txt sort >sorted.txt
*#: % >sorted.txt sort <jumble.txt
*#: % sort < jumble.txt >sorted.txt
*#:: stdout stuff
*# Sort unique lines of jumble.txt into sorted.txt
*#: % <jumble.txt sort | uniq >sorted.txt
*#:: stderr stuff
*# Sort ''HERE doc'' delimited by "lines"
*#: % <<lines sort 
*#::? a second line 
*#::? a first line 
*#::lines a first line a second line stderr stuff

* '''tcsh''' only
*# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
*#: % ( mycmd > /dev/stdin ) > & /dev/null
*#:: stdout stuff
*#: % ( mycmd > /dev/tty ) > & /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: % (mycmd >> out.txt) >& /dev/null
*# Write output to out.txt; store and watch errors ''Note: this happens to be easier in ''tcsh'' - a rare event. Compare the same task in ''sh''. ''
*#: % ( mycmd > out.txt ) | & tee err.txt
*#:: stderr stuff

* '''sh''' only
*# Discard errors, watch output
*#: $ mycmd 2> /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: $ mycmd 2> /dev/null >> out.txt
*# Write output to out.txt; store and watch errors ''Note: this happens to be harder in ''sh'' - a rare event. Compare the same task in ''tcsh''. ''
*#: $ exec 3>&1 ; mycmd 2>&1 >&3 1>out.txt | tee err.txt ; exec 1<&3 3<&-;
*#:: stderr stuff
*# Write messages to err.txt; write output to out.txt and copy output to terminal
*#: $ mycmd 2> err.txt | tee out.txt
*#:: stdout stuff
*# Assign a variable from stored.txt
*#: $ <stored.txt read var; mycmd $var
*#:: stdout stuff
*# Assign first three lines of stored.txt to three different variables
*#: $ exec 3<&0; exec <test; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
*#:: stderr stuff
*#:: stdout stuff
*# Use all unique lines in stored.txt as variable input, appending to result.txt
*#: $ exec 3<&0; exec <test; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
*#:: stderr stuff
*: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 1> out.txt
*:: $ mycmd > out.txt
*::: stderr stuff
*: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 0< in.txt
*:: $ mycmd < in.txt
*::: stderr stuff
*::: stdout stuff
*: ''Note:'' To close a file descriptor, say <code>n<&-</code>
*:: $ mycmd >out.txt 1<&-;
*:: ''-sh: fails without messages''
*:: -bash: mycmd: write error: Bad file descriptor

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to a file. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 > errors.txt
because when you do the first switch, it's done ''right away'' and when the second >& comes around, it's getting the switched data. This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, such as:
samizdata# myprogram 3>&2 2>&1 1>&3 | command
will make the output of ''myprogram'' do this: 3 point to the same place as 2, 2 point to 1, and finally, 1 point to 3 and then pipe all of it to ''command''

[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Redirection

2006-06-27T21:02:43Z

Ninereasons: correct example

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Common operands include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipes and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). The newly assigned process can reassign the open ''fd'' in turn, so that the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

* '''tcsh and sh'''
*# Write output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Append output to a file
*#: % mycmd >> out.txt
*#:: stderr stuff
*# Redirect the output of a remote command to ''local.txt''.
*#: % localcmd "remotecmd" > local.txt
*#:: stderr stuff
*# Same command as above, showing only the changes compared to ''local.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin'. These two commands are equivalent. ''
*#: % localcmd "remotecmd" | diff /dev/stdin local.txt
*#: % localcmd "remotecmd" | diff - local.txt
*#:: stderr stuff
*# Direct <code>stdout</code>+<code>stderr</code> to file
*#: % localcmd >& out.txt
*# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
*#: % <jumble.txt sort >sorted.txt
*#: % >sorted.txt sort <jumble.txt
*#: % sort < jumble.txt >sorted.txt
*#:: stdout stuff
*# Sort unique lines of jumble.txt into sorted.txt
*#: % <jumble.txt sort | uniq >sorted.txt
*#:: stderr stuff
*# Sort ''HERE doc'' delimited by "lines"
*#: % <<lines sort 
*#::? a second line 
*#::? a first line 
*#::lines a first line a second line stderr stuff

* '''tcsh''' only
*# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
*#: % ( mycmd > /dev/stdin ) > & /dev/null
*#:: stdout stuff
*#: % ( mycmd > /dev/tty ) > & /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: % (mycmd >> out.txt) >& /dev/null
*# Write output to out.txt; store and watch errors ''Note: this happens to be easier in ''tcsh'' - a rare event. Compare the same task in ''sh''. ''
*#: % ( mycmd > out.txt ) | & tee err.txt
*#:: stderr stuff

* '''sh''' only
*# Discard errors, watch output
*#: $ mycmd 2> /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: $ mycmd 2> /dev/null >> out.txt
*# Write output to out.txt; store and watch errors ''Note: this happens to be harder in ''sh'' - a rare event. Compare the same task in ''tcsh''. ''
*#: $ exec 3>&1 ; mycmd 2>&1 >&3 1>out.txt | tee err.txt ; exec 1<&3 3<&-;
*#:: stderr stuff
*# Write messages to err.txt; write output to out.txt and copy output to terminal
*#: $ mycmd 2> err.txt | tee out.txt
*#:: stdout stuff
*# Assign a variable from stored.txt
*#: $ <stored.txt read var; mycmd $var
*#:: stdout stuff
*# Assign first three lines of stored.txt to three different variables
*#: $ exec 3<&0; exec <test; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
*#:: stderr stuff
*#:: stdout stuff
*# Use all unique lines in stored.txt as variable input, appending to result.txt
*#: $ exec 3<&0; exec <test; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
*#:: stderr stuff
*: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 1> out.txt
*:: $ mycmd > out.txt
*::: stderr stuff
*: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 0< in.txt
*:: $ mycmd < in.txt
*::: stderr stuff
*::: stdout stuff

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to a file. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 > errors.txt
because when you do the first switch, it's done ''right away'' and when the second >& comes around, it's getting the switched data. This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, such as:
samizdata# myprogram 3>&2 2>&1 1>&3 | command
will make the output of ''myprogram'' do this: 3 point to the same place as 2, 2 point to 1, and finally, 1 point to 3 and then pipe all of it to ''command''

[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Redirection

2006-06-27T20:53:37Z

Ninereasons: change name of file for example

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Common operands include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipes and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). The newly assigned process can reassign the open ''fd'' in turn, so that the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

* '''tcsh and sh'''
*# Write output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Append output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Redirect the output of a remote command to ''local.txt''.
*#: % localcmd "remotecmd" > local.txt
*#:: stderr stuff
*# Same command as above, showing only the changes compared to ''local.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin'. These two commands are equivalent. ''
*#: % localcmd "remotecmd" | diff /dev/stdin local.txt
*#: % localcmd "remotecmd" | diff - local.txt
*#:: stderr stuff
*# Direct <code>stdout</code>+<code>stderr</code> to file
*#: % localcmd >& out.txt
*# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
*#: % <jumble.txt sort >sorted.txt
*#: % >sorted.txt sort <jumble.txt
*#: % sort < jumble.txt >sorted.txt
*#:: stdout stuff
*# Sort unique lines of jumble.txt into sorted.txt
*#: % <jumble.txt sort | uniq >sorted.txt
*#:: stderr stuff
*# Sort ''HERE doc'' delimited by "lines"
*#: % <<lines sort 
*#::? a second line 
*#::? a first line 
*#::lines a first line a second line stderr stuff

* '''tcsh''' only
*# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
*#: % ( mycmd > /dev/stdin ) > & /dev/null
*#:: stdout stuff
*#: % ( mycmd > /dev/tty ) > & /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: % (mycmd >> out.txt) >& /dev/null
*# Write output to out.txt; store and watch errors ''Note: this happens to be easier in ''tcsh'' - a rare event. Compare the same task in ''sh''. ''
*#: % ( mycmd > out.txt ) | & tee err.txt
*#:: stderr stuff

* '''sh''' only
*# Discard errors, watch output
*#: $ mycmd 2> /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: $ mycmd 2> /dev/null >> out.txt
*# Write output to out.txt; store and watch errors ''Note: this happens to be harder in ''sh'' - a rare event. Compare the same task in ''tcsh''. ''
*#: $ exec 3>&1 ; mycmd 2>&1 >&3 1>out.txt | tee err.txt ; exec 1<&3 3<&-;
*#:: stderr stuff
*# Write messages to err.txt; write output to out.txt and copy output to terminal
*#: $ mycmd 2> err.txt | tee out.txt
*#:: stdout stuff
*# Assign a variable from stored.txt
*#: $ <stored.txt read var; mycmd $var
*#:: stdout stuff
*# Assign first three lines of stored.txt to three different variables
*#: $ exec 3<&0; exec <test; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
*#:: stderr stuff
*#:: stdout stuff
*# Use all unique lines in stored.txt as variable input, appending to result.txt
*#: $ exec 3<&0; exec <test; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
*#:: stderr stuff
*: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 1> out.txt
*:: $ mycmd > out.txt
*::: stderr stuff
*: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 0< in.txt
*:: $ mycmd < in.txt
*::: stderr stuff
*::: stdout stuff

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to a file. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 > errors.txt
because when you do the first switch, it's done ''right away'' and when the second >& comes around, it's getting the switched data. This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, such as:
samizdata# myprogram 3>&2 2>&1 1>&3 | command
will make the output of ''myprogram'' do this: 3 point to the same place as 2, 2 point to 1, and finally, 1 point to 3 and then pipe all of it to ''command''

[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Redirection

2006-06-27T20:41:44Z

Ninereasons: equivalent

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Common operands include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipes and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). The newly assigned process can reassign the open ''fd'' in turn, so that the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

* '''tcsh and sh'''
*# Write output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Append output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Redirect the output of a remote command to ''remote.txt''.
*#: % localcmd "remotecmd" > remote.txt
*#:: stderr stuff
*# Same command as above, showing only the changes compared to ''remote.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin'. These two commands are equivalent. ''
*#: % localcmd "remotecmd" | diff /dev/stdin remote.txt
*#: % localcmd "remotecmd" | diff - remote.txt
*#:: stderr stuff
*# Direct <code>stdout</code>+<code>stderr</code> to file
*#: % localcmd >& out.txt
*# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
*#: % <jumble.txt sort >sorted.txt
*#: % >sorted.txt sort <jumble.txt
*#: % sort < jumble.txt >sorted.txt
*#:: stdout stuff
*# Sort unique lines of jumble.txt into sorted.txt
*#: % <jumble.txt sort | uniq >sorted.txt
*#:: stderr stuff
*# Sort ''HERE doc'' delimited by "lines"
*#: % <<lines sort 
*#::? a second line 
*#::? a first line 
*#::lines a first line a second line stderr stuff

* '''tcsh''' only
*# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
*#: % ( mycmd > /dev/stdin ) > & /dev/null
*#:: stdout stuff
*#: % ( mycmd > /dev/tty ) > & /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: % (mycmd >> out.txt) >& /dev/null
*# Write output to out.txt; store and watch errors ''Note: this happens to be easier in ''tcsh'' - a rare event. Compare the same task in ''sh''. ''
*#: % ( mycmd > out.txt ) | & tee err.txt
*#:: stderr stuff

* '''sh''' only
*# Discard errors, watch output
*#: $ mycmd 2> /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: $ mycmd 2> /dev/null >> out.txt
*# Write output to out.txt; store and watch errors ''Note: this happens to be harder in ''sh'' - a rare event. Compare the same task in ''tcsh''. ''
*#: $ exec 3>&1 ; mycmd 2>&1 >&3 1>out.txt | tee err.txt ; exec 1<&3 3<&-;
*#:: stderr stuff
*# Write messages to err.txt; write output to out.txt and copy output to terminal
*#: $ mycmd 2> err.txt | tee out.txt
*#:: stdout stuff
*# Assign a variable from stored.txt
*#: $ <stored.txt read var; mycmd $var
*#:: stdout stuff
*# Assign first three lines of stored.txt to three different variables
*#: $ exec 3<&0; exec <test; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
*#:: stderr stuff
*#:: stdout stuff
*# Use all unique lines in stored.txt as variable input, appending to result.txt
*#: $ exec 3<&0; exec <test; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
*#:: stderr stuff
*: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 1> out.txt
*:: $ mycmd > out.txt
*::: stderr stuff
*: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 0< in.txt
*:: $ mycmd < in.txt
*::: stderr stuff
*::: stdout stuff

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to a file. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 > errors.txt
because when you do the first switch, it's done ''right away'' and when the second >& comes around, it's getting the switched data. This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, such as:
samizdata# myprogram 3>&2 2>&1 1>&3 | command
will make the output of ''myprogram'' do this: 3 point to the same place as 2, 2 point to 1, and finally, 1 point to 3 and then pipe all of it to ''command''

[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Redirection

2006-06-27T20:37:48Z

Ninereasons: /* Redirection in <code>sh</code> compared to <code>tcsh</code> */ - add prompts

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Common operands include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipes and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). The newly assigned process can reassign the open ''fd'' in turn, so that the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

* '''tcsh and sh'''
*# Write output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Append output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Redirect the output of a remote command to ''remote.txt''.
*#: % localcmd "remotecmd" > remote.txt
*#:: stderr stuff
*# Same command as above, showing only the changes compared to ''remote.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin' ''
*#: % localcmd "remotecmd" | diff /dev/stdin remote.txt
*#:: stderr stuff
*#: % localcmd "remotecmd" | diff - remote.txt
*#:: stderr stuff
*# Direct <code>stdout</code>+<code>stderr</code> to file
*#: % localcmd >& out.txt
*# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
*#: % <jumble.txt sort >sorted.txt
*#: % >sorted.txt sort <jumble.txt
*#: % sort < jumble.txt >sorted.txt
*#:: stdout stuff
*# Sort unique lines of jumble.txt into sorted.txt
*#: % <jumble.txt sort | uniq >sorted.txt
*#:: stderr stuff
*# Sort ''HERE doc'' delimited by "lines"
*#: % <<lines sort 
*#::? a second line 
*#::? a first line 
*#::lines a first line a second line stderr stuff

* '''tcsh''' only
*# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
*#: % ( mycmd > /dev/stdin ) > & /dev/null
*#:: stdout stuff
*#: % ( mycmd > /dev/tty ) > & /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: % (mycmd >> out.txt) >& /dev/null
*# Write output to out.txt; store and watch errors ''Note: this happens to be easier in ''tcsh'' - a rare event. Compare the same task in ''sh''. ''
*#: % ( mycmd > out.txt ) | & tee err.txt
*#:: stderr stuff

* '''sh''' only
*# Discard errors, watch output
*#: $ mycmd 2> /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: $ mycmd 2> /dev/null >> out.txt
*# Write output to out.txt; store and watch errors ''Note: this happens to be harder in ''sh'' - a rare event. Compare the same task in ''tcsh''. ''
*#: $ exec 3>&1 ; mycmd 2>&1 >&3 1>out.txt | tee err.txt ; exec 1<&3 3<&-;
*#:: stderr stuff
*# Write messages to err.txt; write output to out.txt and copy output to terminal
*#: $ mycmd 2> err.txt | tee out.txt
*#:: stdout stuff
*# Assign a variable from stored.txt
*#: $ <stored.txt read var; mycmd $var
*#:: stdout stuff
*# Assign first three lines of stored.txt to three different variables
*#: $ exec 3<&0; exec <test; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
*#:: stderr stuff
*#:: stdout stuff
*# Use all unique lines in stored.txt as variable input, appending to result.txt
*#: $ exec 3<&0; exec <test; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
*#:: stderr stuff
*: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 1> out.txt
*:: $ mycmd > out.txt
*::: stderr stuff
*: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 0< in.txt
*:: $ mycmd < in.txt
*::: stderr stuff
*::: stdout stuff

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to a file. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 > errors.txt
because when you do the first switch, it's done ''right away'' and when the second >& comes around, it's getting the switched data. This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, such as:
samizdata# myprogram 3>&2 2>&1 1>&3 | command
will make the output of ''myprogram'' do this: 3 point to the same place as 2, 2 point to 1, and finally, 1 point to 3 and then pipe all of it to ''command''

[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Redirection

2006-06-27T20:36:04Z

Ninereasons: generalize example

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Common operands include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipes and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). The newly assigned process can reassign the open ''fd'' in turn, so that the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

* '''tcsh and sh'''
*# Write output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Append output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Redirect the output of a remote command to ''remote.txt''.
*#: % localcmd "remotecmd" > remote.txt
*#:: stderr stuff
*# Same command as above, showing only the changes compared to ''remote.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin' ''
*#: % localcmd "remotecmd" | diff /dev/stdin remote.txt
*#:: stderr stuff
*#: % localcmd "remotecmd" | diff - remote.txt
*#:: stderr stuff
*# Direct <code>stdout</code>+<code>stderr</code> to file
*#: % localcmd >& out.txt
*# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
*#: % <jumble.txt sort >sorted.txt
*#: % >sorted.txt sort <jumble.txt
*#: % sort < jumble.txt >sorted.txt
*#:: stdout stuff
*# Sort unique lines of jumble.txt into sorted.txt
*#: % <jumble.txt sort | uniq >sorted.txt
*#:: stderr stuff
*# Sort ''HERE doc'' delimited by "lines"
*#: % <<lines sort 
*#::? a second line 
*#::? a first line 
*#::lines a first line a second line stderr stuff

* '''tcsh''' only
*# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
*#: % ( mycmd > /dev/stdin ) > & /dev/null
*#:: stdout stuff
*#: % ( mycmd > /dev/tty ) > & /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: % (mycmd >> out.txt) >& /dev/null
*# Write output to out.txt; store and watch errors ''Note: this happens to be easier in ''tcsh'' - a rare event. Compare the same task in ''sh''. ''
*#: % ( mycmd > out.txt ) | & tee err.txt
*#:: stderr stuff

* '''sh''' only
*# Discard errors, watch output
*#: $ mycmd 2> /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: $ mycmd 2> /dev/null >> out.txt
*# Write output to out.txt; store and watch errors ''Note: this happens to be harder in ''sh'' - a rare event. Compare the same task in ''tcsh''. ''
*#: $ exec 3>&1 ; mycmd 2>&1 >&3 1>out.txt | tee err.txt ; exec 1<&3 3<&-;
*#:: stderr stuff
*# Write messages to err.txt; write output to out.txt and copy output to terminal
*#: $ mycmd 2> err.txt | tee out.txt
*#:: stdout stuff
*# Assign a variable from stored.txt
*#: $ <stored.txt read var; mycmd $var
*#:: stdout stuff
*# Assign first three lines of stored.txt to three different variables
*#: $ exec 3<&0; exec <test; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
*#:: stderr stuff
*#:: stdout stuff
*# Use all unique lines in stored.txt as variable input, appending to result.txt
*#: exec 3<&0; exec <test; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
*#:: stderr stuff
*: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 1> out.txt
*:: $ mycmd > out.txt
*::: stderr stuff
*: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 0< in.txt
*:: $ mycmd < in.txt
*::: stderr stuff
*::: stdout stuff

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to a file. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 > errors.txt
because when you do the first switch, it's done ''right away'' and when the second >& comes around, it's getting the switched data. This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, such as:
samizdata# myprogram 3>&2 2>&1 1>&3 | command
will make the output of ''myprogram'' do this: 3 point to the same place as 2, 2 point to 1, and finally, 1 point to 3 and then pipe all of it to ''command''

[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Redirection

2006-06-27T20:33:07Z

Ninereasons: correct example

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Common operands include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipes and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). The newly assigned process can reassign the open ''fd'' in turn, so that the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

* '''tcsh and sh'''
*# Write output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Append output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Redirect the output of a remote command to ''remote.txt''.
*#: % localcmd "remotecmd" > remote.txt
*#:: stderr stuff
*# Same command as above, showing only the changes compared to ''remote.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin' ''
*#: % localcmd "remotecmd" | diff /dev/stdin remote.txt
*#:: stderr stuff
*#: % localcmd "remotecmd" | diff - remote.txt
*#:: stderr stuff
*# Direct <code>stdout</code>+<code>stderr</code> to file
*#: % localcmd >& out.txt
*# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
*#: % <jumble.txt sort >sorted.txt
*#: % >sorted.txt sort <jumble.txt
*#: % sort < jumble.txt >sorted.txt
*#:: stdout stuff
*# Sort unique lines of jumble.txt into sorted.txt
*#: % <jumble.txt sort | uniq >sorted.txt
*#:: stderr stuff
*# Sort ''HERE doc'' delimited by "lines"
*#: % <<lines sort 
*#::? a second line 
*#::? a first line 
*#::lines a first line a second line stderr stuff

* '''tcsh''' only
*# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
*#: % ( mycmd > /dev/stdin ) > & /dev/null
*#:: stdout stuff
*#: % ( mycmd > /dev/tty ) > & /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: % (mycmd >> out.txt) >& /dev/null
*# Write output to out.txt; store and watch errors ''Note: this happens to be easier in ''tcsh'' - a rare event. Compare the same task in ''sh''. ''
*#: % ( mycmd > out.txt ) | & tee err.txt
*#:: stderr stuff

* '''sh''' only
*# Discard errors, watch output
*#: $ mycmd 2> /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: $ mycmd 2> /dev/null >> out.txt
*# Write output to out.txt; store and watch errors ''Note: this happens to be harder in ''sh'' - a rare event. Compare the same task in ''tcsh''. ''
*#: $ exec 3>&1 ; tiffinfo 001.tif 2>&1 >&3 1>001.out | tee 001.err ; exec 1<&3 3<&-;
*#:: stderr stuff
*# Write messages to err.txt; write output to out.txt and copy output to terminal
*#: $ mycmd 2> err.txt | tee out.txt
*#:: stdout stuff
*# Assign a variable from stored.txt
*#: $ <stored.txt read var; mycmd $var
*#:: stdout stuff
*# Assign first three lines of stored.txt to three different variables
*#: $ exec 3<&0; exec <test; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
*#:: stderr stuff
*#:: stdout stuff
*# Use all unique lines in stored.txt as variable input, appending to result.txt
*#: exec 3<&0; exec <test; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
*#:: stderr stuff
*: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 1> out.txt
*:: $ mycmd > out.txt
*::: stderr stuff
*: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 0< in.txt
*:: $ mycmd < in.txt
*::: stderr stuff
*::: stdout stuff

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to a file. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 > errors.txt
because when you do the first switch, it's done ''right away'' and when the second >& comes around, it's getting the switched data. This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, such as:
samizdata# myprogram 3>&2 2>&1 1>&3 | command
will make the output of ''myprogram'' do this: 3 point to the same place as 2, 2 point to 1, and finally, 1 point to 3 and then pipe all of it to ''command''

[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Redirection

2006-06-27T20:24:48Z

Ninereasons: /* Redirection in <code>sh</code> compared to <code>tcsh</code> */ - add prompts

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Common operands include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipes and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). The newly assigned process can reassign the open ''fd'' in turn, so that the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

* '''tcsh and sh'''
*# Write output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Append output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Redirect the output of a remote command to ''remote.txt''.
*#: % localcmd "remotecmd" > remote.txt
*#:: stderr stuff
*# Same command as above, showing only the changes compared to ''remote.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin' ''
*#: % localcmd "remotecmd" | diff /dev/stdin remote.txt
*#:: stderr stuff
*#: % localcmd "remotecmd" | diff - remote.txt
*#:: stderr stuff
*# Direct <code>stdout</code>+<code>stderr</code> to file
*#: % localcmd >& out.txt
*# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
*#: % <jumble.txt sort >sorted.txt
*#: % >sorted.txt sort <jumble.txt
*#: % sort < jumble.txt >sorted.txt
*#:: stdout stuff
*# Sort unique lines of jumble.txt into sorted.txt
*#: % <jumble.txt sort | uniq >sorted.txt
*#:: stderr stuff
*# Sort ''HERE doc'' delimited by "lines"
*#: % <<lines sort 
*#::? a second line 
*#::? a first line 
*#::lines a first line a second line stderr stuff

* '''tcsh''' only
*# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
*#: % ( mycmd > /dev/stdin ) > & /dev/null
*#:: stdout stuff
*#: % ( mycmd > /dev/tty ) > & /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: % (mycmd >> out.txt) >& /dev/null
*# Write output to out.txt; store and watch errors ''Note: this happens to be easier in ''tcsh'' - a rare event. Compare the same task in ''sh''. ''
*#: % ( mycmd > out.txt ) | & tee err.txt
*#:: stderr stuff

* '''sh''' only
*# Discard errors, watch output
*#: $ mycmd 2> /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: $ mycmd 2> /dev/null > out.txt
*# Write output to out.txt; store and watch errors ''Note: this happens to be harder in ''sh'' - a rare event. Compare the same task in ''tcsh''. ''
*#: $ exec 3>&1 ; tiffinfo 001.tif 2>&1 >&3 1>001.out | tee 001.err ; exec 1<&3 3<&-;
*#:: stderr stuff
*# Write messages to err.txt; write output to out.txt and copy output to terminal
*#: $ mycmd 2> err.txt | tee out.txt
*#:: stdout stuff
*# Assign a variable from stored.txt
*#: $ <stored.txt read var; mycmd $var
*#:: stdout stuff
*# Assign first three lines of stored.txt to three different variables
*#: $ exec 3<&0; exec <test; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
*#:: stderr stuff
*#:: stdout stuff
*# Use all unique lines in stored.txt as variable input, appending to result.txt
*#: exec 3<&0; exec <test; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
*#:: stderr stuff
*: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 1> out.txt
*:: $ mycmd > out.txt
*::: stderr stuff
*: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 0< in.txt
*:: $ mycmd < in.txt
*::: stderr stuff
*::: stdout stuff

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to a file. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 > errors.txt
because when you do the first switch, it's done ''right away'' and when the second >& comes around, it's getting the switched data. This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, such as:
samizdata# myprogram 3>&2 2>&1 1>&3 | command
will make the output of ''myprogram'' do this: 3 point to the same place as 2, 2 point to 1, and finally, 1 point to 3 and then pipe all of it to ''command''

[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Redirection

2006-06-27T20:18:38Z

Ninereasons: /* Redirection in <code>sh</code> compared to <code>tcsh</code> */ - add prompts

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Common operands include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipes and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). The newly assigned process can reassign the open ''fd'' in turn, so that the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

* '''tcsh and sh'''
*# Write output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Append output to a file
*#: % mycmd > out.txt
*#:: stderr stuff
*# Redirect the output of a remote command to ''remote.txt''.
*#: % localcmd "remotecmd" > remote.txt
*#:: stderr stuff
*# Same command as above, showing only the changes compared to ''remote.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin' ''
*#: % localcmd "remotecmd" | diff /dev/stdin remote.txt
*#:: stderr stuff
*#: % localcmd "remotecmd" | diff - remote.txt
*#:: stderr stuff
*# Direct <code>stdout</code>+<code>stderr</code> to file
*#: % localcmd >& out.txt
*# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
*#: % <jumble.txt sort >sorted.txt
*#: % >sorted.txt sort <jumble.txt
*#: % sort < jumble.txt >sorted.txt
*#:: stdout stuff
*# Sort unique lines of jumble.txt into sorted.txt
*#: % <jumble.txt sort | uniq >sorted.txt
*#:: stderr stuff
*# Sort ''HERE doc'' delimited by "lines"
*#: % <<lines sort 
*#::? a second line 
*#::? a first line 
*#::lines a first line a second line stderr stuff

* '''tcsh''' only
*# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
*#: % ( mycmd > /dev/stdin ) > & /dev/null
*#:: stdout stuff
*#: % ( mycmd > /dev/tty ) > & /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: % (mycmd >> out.txt) >& /dev/null
*# Write output to out.txt; store and watch errors ''Note: this happens to be easier in ''tcsh'' - a rare event. Compare the same task in ''sh''. ''
*#: % ( mycmd > out.txt ) | & tee err.txt
*#:: stderr stuff

* '''sh''' only
*# Discard errors, watch output
*#: $ mycmd 2> /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: $ mycmd 2> /dev/null > out.txt
*# Write output to out.txt; store and watch errors ''Note: this happens to be harder in ''sh'' - a rare event. Compare the same task in ''tcsh''. ''
*#: exec 3>&1 ; tiffinfo 001.tif 2>&1 >&3 1>001.out | tee 001.err ; exec 1<&3 3<&-;
*#:: stderr stuff
*# Write messages to err.txt; write output to out.txt and copy output to terminal
*#: $ mycmd 2> err.txt | tee out.txt
*#:: stdout stuff
*# Assign a variable from stored.txt
*#: $ <stored.txt read var; mycmd $var
*#:: stdout stuff
*# Assign first three lines of stored.txt to three different variables
*#: $ exec 3<&0; exec <test; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
*#:: stderr stuff
*#:: stdout stuff
*# Use all unique lines in stored.txt as variable input, appending to result.txt
*#: exec 3<&0; exec <test; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
*#:: stderr stuff
*: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 1> out.txt
*:: $ mycmd > out.txt
*::: stderr stuff
*: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 0< in.txt
*:: $ mycmd < in.txt
*::: stderr stuff
*::: stdout stuff

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to a file. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 > errors.txt
because when you do the first switch, it's done ''right away'' and when the second >& comes around, it's getting the switched data. This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, such as:
samizdata# myprogram 3>&2 2>&1 1>&3 | command
will make the output of ''myprogram'' do this: 3 point to the same place as 2, 2 point to 1, and finally, 1 point to 3 and then pipe all of it to ''command''

[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Redirection

2006-06-27T20:15:02Z

Ninereasons: /* Redirection in bash (or sh) */ - more detail and examples

'''Redirection''' is sending the output of a program to somewhere other than where it would otherwise go - for example you can '''redirect''' the output of an [[ls]] command to a text file for later processing or to the [[grep]] command for filtering. Common operands include: '''>''', '''>>''', '''<''', '''<<''', and the ever popular '''|'''.

> sends output to a file (may include [[special files]] such as [[/dev/null]])
>> appends output to a file (without overwriting it)
< read file to <code>stdin</code>
<< read to stdin from <code><<delimiter</code> to <code>delimiter</code> (a ''HERE doc'').
| sends output to a program (frequently, a [[:Category:System Commands|system command]] like [[grep]])

If you're using the [[bash]] or [[bourne shell|bourne]] shells, you also have some special options available to you: you can redirect standard input, standard output and standard error messages with far greater flexibility and reliability. Other shells such as [[csh]] are notably limited in redirection capability, making them better suited to interactive use than to [[shell programming]] or other complex uses.

== Shell pipes and redirection ==
There are three standard input/output file descriptors (''fd'') that are preconnected to the shell process running on your FreeBSD machine. Most commands that you would run from the command line expect these file descriptors to be open and accessible. The first (''fd 0'') provides a stream of ''input'' to your program, the second (''fd 1'') provides a stream of ''output'', and the third (''fd 2'') provides a stream of diagnostic messages (usually to your terminal).

When you open a terminal, before running the shell the terminal device is opened three times to preconnect these file descriptors. The shell then inherits the file descriptors, and passes them on to each process run from the shell.

On FreeBSD systems it looks like this:
{| border="1" cellpadding="5" cellspacing="0"
|-
! file descriptor || Stream || file descriptor file || device path
|-
| '''0''' || Standard input || <code>/dev/stdin</code> || /dev/fd/0
|-
| '''1''' || Standard output || <code>/dev/stdout</code> || /dev/fd/1
|-
| '''2''' || Standard error || <code>/dev/stderr</code> || /dev/fd/2
|}

The example below demonstrates that by default, all of these input/output streams are directed to your terminal (color is added).
$ for i in stdin stdout stderr; do echo $i stuff > /dev/$i; done
 stdin stuff
stdout stuff
stderr stuff

Redirection means that, the file descriptor is temporarily reassigned to somewhere other than the terminal device (a file, a pipe, another file descriptor). The newly assigned process can reassign the open ''fd'' in turn, so that the stream of data is passed along, for example, by writing to <code>stdout</code>, which has been temporarily reassigned to the <code>stdin</code> for the next process by means of a ''pipe'' ('''|'''). This handing off of open file descriptors by the shell, from one sub-process to another, is called a "shell pipeline".

There are ''fd n'' (3-9) descriptors potentially available, but only 0-2 are preconnected to the shell; the others must be created from the shell. In some shells, a standard file descriptor can be detached and reassigned to another file descriptor for as long as the terminal device is open, using the [[exec]] command (examples appear below). This capability is just one case among many of why <code>sh</code> is useful for scripting, and <code>tcsh</code> (which lacks it) is not (''see the [http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ FAQ]).

== Redirection in <code>sh</code> compared to <code>tcsh</code> ==
The c-shells (<code>tcsh</code> or <code>csh</code>) and the Bourne shells (<code>sh</code> or <code>bash</code>) do not handle redirection or piping in quite the same way.

* '''tcsh and sh'''
*# Write output to a file
*#: mycmd > out.txt
*#:: stderr stuff
*# Append output to a file
*#: mycmd > out.txt
*#:: stderr stuff
*# Redirect the output of a remote command to ''remote.txt''.
*#: % localcmd "remotecmd" > remote.txt
*#:: stderr stuff
*# Same command as above, showing only the changes compared to ''remote.txt''. ''Note: Many programs recognize '-' as a shortcut for '/dev/stdin' ''
*#: % localcmd "remotecmd" | diff /dev/stdin remote.txt
*#:: stderr stuff
*#: % localcmd "remotecmd" | diff - remote.txt
*#:: stderr stuff
*# Direct <code>stdout</code>+<code>stderr</code> to file
*#: % localcmd >& out.txt
*# Sort lines of jumble.txt into sorted.txt ''Note: the sequence in which redirection appears is not important. All of the following are exactly equivalent.''
*#: % <jumble.txt sort >sorted.txt
*#: % >sorted.txt sort <jumble.txt
*#: % sort < jumble.txt >sorted.txt
*#:: stdout stuff
*# Sort unique lines of jumble.txt into sorted.txt
*#: % <jumble.txt sort | uniq >sorted.txt
*#:: stderr stuff
*# Sort ''HERE doc'' delimited by "lines"
*#: % <<lines sort 
*#::? a second line 
*#::? a first line 
*#::lines a first line a second line stderr stuff

* '''tcsh''' only
*# Discard errors, watch output (probably evil) ''Note: There is no reliable way to do this in tcsh. Here we exploit the fact that terminal reads from <code>stdin</code>. ''
*#: % ( mycmd > /dev/stdin ) > & /dev/null
*#:: stdout stuff
*#: % ( mycmd > /dev/tty ) > & /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: % (mycmd >> out.txt) >& /dev/null
*# Write output to out.txt; store and watch errors ''Note: this happens to be easier in ''tcsh'' - a rare event. Compare the same task in ''sh''. ''
*#: % ( mycmd > out.txt ) | & tee err.txt
*#:: stderr stuff

* '''sh''' only
*# Discard errors, watch output
*#: $ mycmd 2> /dev/null
*#:: stdout stuff
*# Append output to out.txt; discard messages
*#: $ mycmd 2> /dev/null > out.txt
*# Write output to out.txt; store and watch errors ''Note: this happens to be harder in ''sh'' - a rare event. Compare the same task in ''tcsh''. ''
*#: exec 3>&1 ; tiffinfo 001.tif 2>&1 >&3 1>001.out | tee 001.err ; exec 1<&3 3<&-;
*#:: stderr stuff
*# Write messages to err.txt; write output to out.txt and copy output to terminal
*#: $ mycmd 2> err.txt | tee out.txt
*#:: stdout stuff
*# Assign a variable from stored.txt
*#: $ <stored.txt read var; mycmd $var
*#:: stdout stuff
*# Assign first three lines of stored.txt to three different variables
*#: $ exec 3<&0; exec <test; read v1; read v2; read v3; exec 0<&3 3<&-; echo $v1 $v2 $v3
*#:: stderr stuff
*#:: stdout stuff
*# Use all unique lines in stored.txt as variable input, appending to result.txt
*#: exec 3<&0; exec <test; sort | uniq | while read line; do mycmd $line >> result.txt ; done; exec 0<&3 3<&-;
*#:: stderr stuff
*: ''Note:'' By default, redirection pointed right represents <code>stdout</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 1> out.txt
*:: $ mycmd > out.txt
*::: stderr stuff
*: ''Note:'' By default, redirection pointed left represents <code>stdin</code>, so that these two commands are exactly equivalent:
*:: $ mycmd 0< in.txt
*:: $ mycmd < in.txt
*::: stderr stuff
*::: stdout stuff

=== A little more about ''fd n'' in <code>sh</code> ===

Let's say you want to send output to your screen and errors to a file. You '''can't''' just do
samizdata# myprogram 1>&2 2>&1 > errors.txt
because when you do the first switch, it's done ''right away'' and when the second >& comes around, it's getting the switched data. This is where the other, normally unused, file descriptors 3-9 come in. You can use them as place-holders, such as:
samizdata# myprogram 3>&2 2>&1 1>&3 | command
will make the output of ''myprogram'' do this: 3 point to the same place as 2, 2 point to 1, and finally, 1 point to 3 and then pipe all of it to ''command''

[[Category:FreeBSD Terminology]] [[Category:Common Tasks]]

Redirection

2006-06-27T17:34:50Z

Ninereasons: edit description of redirectors

PHP

2006-06-23T00:35:07Z

Ninereasons: /* Using PHP */

PHP is the recursive acronym for ''PHP Hypertext Preprocessor''. It is an interpreted script language commonly used for dynamic pages generation on webservers. Then it is generaly installed side by side with the [[Apache]] web server, and [[MySQL]] or [[PostgreSQL]] as database management system.

==Installing PHP==

PHP is available trougth the port-tree ...
# cd /usr/ports/lang/php5
... and the packages ...
# pkg_add -r php5

==Using PHP==
There are two ways to use PHP :
* With apache: configure apache to use php to process .php files requested by the user.
* As a command interpreter: php can process scripts, or instructions typed at the commandline.

===PHP for dynamic websites===

All the steps of Apache configuration are explained after PHP installation. You have to add these two lines in httpd.conf:
AddType application/x-httpd-php .php
AddType application/x-httpd-php-source .phps

You will probably also want to add ''index.php'' as a possible directory index:
DirectoryIndex index.html index.php

You have to restart Apache to have your changes effective. For example, if you use Apache 2:
/usr/local/etc/rc.d/apache2 restart

Now, you can try your installation by creating a file test.php:
<?
phpinfo();
?>

Request test.php using your web browser, and if all is well it will show you your PHP configuration.

===Scripts in PHP===

If you have compiled PHP with the commandline interface, you can write scripts in PHP that can be used from the [[shell]]. This is very useful for ''quick and dirty'' hacks; especially if you want your output to be HTML. Put the PHP code in a file like this :
<pre>
#!/usr/local/bin/php
# phpscript
<html> <title>Defined Constants</title> <body>

<h1><? echo "Hello from PHP " .phpversion() ?></h1>
<pre><? print_r(get_defined_constants()) ?></pre>

</body></html>
</pre>

Make the file executable:
'''%''' chmod +x phpscript

Then, you can run it from the commandline.

'''%''' ./phpscript | lynx -stdin
In this case, we've piped to <code>lynx</code>, so that we can browse the nicely formatted result.
<pre>
Defined Constants (p1 of 154)

Hello from PHP 5.1.4

Array
(
[E_ERROR] => 1
[E_WARNING] => 2
[E_PARSE] => 4
[E_NOTICE] => 8
-- press space for more, use arrow keys to move, '?' for help, 'q' to quit.
</pre>

Or, we could have executed the command directly from the commandline:
'''%''' php -r 'echo "Hello from PHP ". phpversion() . "\n"; print_r(get_defined_constants());'
Hello from PHP 5.1.4
Array
(
...

==PHP Website==
[http://www.php.net http://www.php.net]

[[Category: Ports and Packages]]
[[Category: Configuring FreeBSD]]

PHP

2006-06-23T00:34:13Z

Ninereasons: /* PHP for dynamic websites */

PHP is the recursive acronym for ''PHP Hypertext Preprocessor''. It is an interpreted script language commonly used for dynamic pages generation on webservers. Then it is generaly installed side by side with the [[Apache]] web server, and [[MySQL]] or [[PostgreSQL]] as database management system.

==Installing PHP==

PHP is available trougth the port-tree ...
# cd /usr/ports/lang/php5
... and the packages ...
# pkg_add -r php5

==Using PHP==
There are two ways to use PHP :
* With apache: configure apache to use php to process .php files requested by the user.
* As command interpreter: php can process scripts, or instructions typed at the commandline.

===PHP for dynamic websites===

All the steps of Apache configuration are explained after PHP installation. You have to add these two lines in httpd.conf:
AddType application/x-httpd-php .php
AddType application/x-httpd-php-source .phps

You will probably also want to add ''index.php'' as a possible directory index:
DirectoryIndex index.html index.php

You have to restart Apache to have your changes effective. For example, if you use Apache 2:
/usr/local/etc/rc.d/apache2 restart

Now, you can try your installation by creating a file test.php:
<?
phpinfo();
?>

Request test.php using your web browser, and if all is well it will show you your PHP configuration.

===Scripts in PHP===

If you have compiled PHP with the commandline interface, you can write scripts in PHP that can be used from the [[shell]]. This is very useful for ''quick and dirty'' hacks; especially if you want your output to be HTML. Put the PHP code in a file like this :
<pre>
#!/usr/local/bin/php
# phpscript
<html> <title>Defined Constants</title> <body>

<h1><? echo "Hello from PHP " .phpversion() ?></h1>
<pre><? print_r(get_defined_constants()) ?></pre>

</body></html>
</pre>

Make the file executable:
'''%''' chmod +x phpscript

Then, you can run it from the commandline.

'''%''' ./phpscript | lynx -stdin
In this case, we've piped to <code>lynx</code>, so that we can browse the nicely formatted result.
<pre>
Defined Constants (p1 of 154)

Hello from PHP 5.1.4

Array
(
[E_ERROR] => 1
[E_WARNING] => 2
[E_PARSE] => 4
[E_NOTICE] => 8
-- press space for more, use arrow keys to move, '?' for help, 'q' to quit.
</pre>

Or, we could have executed the command directly from the commandline:
'''%''' php -r 'echo "Hello from PHP ". phpversion() . "\n"; print_r(get_defined_constants());'
Hello from PHP 5.1.4
Array
(
...

==PHP Website==
[http://www.php.net http://www.php.net]

[[Category: Ports and Packages]]
[[Category: Configuring FreeBSD]]

PHP

2006-06-23T00:33:26Z

Ninereasons: /* Using PHP */

PHP is the recursive acronym for ''PHP Hypertext Preprocessor''. It is an interpreted script language commonly used for dynamic pages generation on webservers. Then it is generaly installed side by side with the [[Apache]] web server, and [[MySQL]] or [[PostgreSQL]] as database management system.

==Installing PHP==

PHP is available trougth the port-tree ...
# cd /usr/ports/lang/php5
... and the packages ...
# pkg_add -r php5

==Using PHP==
There are two ways to use PHP :
* With apache: configure apache to use php to process .php files requested by the user.
* As command interpreter: php can process scripts, or instructions typed at the commandline.

===PHP for dynamic websites===

All the steps of Apache configuration are explained after PHP installation. You have to add these two lines in httpd.conf:
AddType application/x-httpd-php .php
AddType application/x-httpd-php-source .phps

You will probably also want to add ''index.php'' as a possible directory index:
DirectoryIndex index.html index.php

You have to restart Apache to have your changes effective. For example, if you use Apache 2:
/usr/local/etc/rc.d/apache2 restart

Now, you can try your installation by creating a file test.php:
<?
phpinfo();
?>

Request test.php using your web browser, and if all is well it will show you your php settings.

===Scripts in PHP===

If you have compiled PHP with the commandline interface, you can write scripts in PHP that can be used from the [[shell]]. This is very useful for ''quick and dirty'' hacks; especially if you want your output to be HTML. Put the PHP code in a file like this :
<pre>
#!/usr/local/bin/php
# phpscript
<html> <title>Defined Constants</title> <body>

<h1><? echo "Hello from PHP " .phpversion() ?></h1>
<pre><? print_r(get_defined_constants()) ?></pre>

</body></html>
</pre>

Make the file executable:
'''%''' chmod +x phpscript

Then, you can run it from the commandline.

'''%''' ./phpscript | lynx -stdin
In this case, we've piped to <code>lynx</code>, so that we can browse the nicely formatted result.
<pre>
Defined Constants (p1 of 154)

Hello from PHP 5.1.4

Array
(
[E_ERROR] => 1
[E_WARNING] => 2
[E_PARSE] => 4
[E_NOTICE] => 8
-- press space for more, use arrow keys to move, '?' for help, 'q' to quit.
</pre>

Or, we could have executed the command directly from the commandline:
'''%''' php -r 'echo "Hello from PHP ". phpversion() . "\n"; print_r(get_defined_constants());'
Hello from PHP 5.1.4
Array
(
...

==PHP Website==
[http://www.php.net http://www.php.net]

[[Category: Ports and Packages]]
[[Category: Configuring FreeBSD]]

Regular expressions

2006-06-22T18:39:20Z

Ninereasons: egrep

'''Regular expressions''' are patterns that are used to describe a string of characters. Regular expressions might be used to match substrings in text as a logical condition for doing something in a program, for capturing the text matched by the expression in order to do something with it, and many other uses. Users more familiar with Windows might think of regular expressions as comparable to "wildcards", only wilder.

From the commandline and in [[shell script]]s, the stream editor ([[sed]]) is used to make substitutions for matched text. <code>egrep</code> (shorthand for <code>grep -E</code>) filters strings using regular expressions. For more complex types of pattern matching and text manipulation, [[Perl]] and [[Tcl]] have a built-in regular expression engine; and most modern programming languages have regular expression modules.

== external links ==
[http://en.wikipedia.org/wiki/Regular_expression Wikipedia entry]

[[Category:FreeBSD Terminology]]

Talk:DIR

2006-06-21T18:44:20Z

Ninereasons: /* grep > simple wildcarding */

==Best in what sense?==

:''ls is best combined with grep if you plan on doing any filtering: dir *.txt is equivalent to ls | grep .txt, and dir boogy*.* is equivalent to ls | grep boogy.''

What is "best" about this? [[User:Ninereasons|Ninereasons]] 20:02, 19 June 2006 (EDT)

== grep > simple wildcarding ==

"best" in the sense that you get far more fine grained control out of piping to grep than you would with the use of simple wildcards.

: Ok. But how do the examples illustrate this? [[User:Ninereasons|Ninereasons]] 14:44, 21 June 2006 (EDT)

Daemon

2006-06-20T21:43:08Z

Ninereasons: /* Prevent daemonic mischief */ more daemonic scariness

A '''daemon''' is a program that runs in the background, rather than under the direct control of a user. If you are more familiar with DOS or early Windows, this kind of program would be known to you as a ''TSR'' (''Terminate and Stay Resident'') program. In NT systems, you would refer to these as ''[[services]]''. Unix users might use the name ''daemon'', ''[[background processes]]'' and ''service'' with rough equivalence, especially when speaking to a general audience.

In FreeBSD, programs that are run as daemons will (usually) have a "d" appended to their name, e.g., [[sshd]], [[named]], etc. Loosely speaking, a ''daemon'' is any process that runs in the background (instead of interactively). If an interactive process such as a shell is used to start the ''daemon'', the ''daemon'' continues to run after the interactive process is closed. In FreeBSD, ''daemons'' are ordinarily started by [[init]] at bootup, and stopped by [[init]] at shutdown.

The name comes from the old Greek (daimon, ''δαιμων''), referring to inferior spirits that do things the gods would rather not be concerned with - like do work.

==Prevent daemonic mischief==
It matters when and how a ''daemon'' is started. Carelessly started, a ''daemon'' might leak information to the world about your username and other information or capabilities exported from your login environment, provide the world with access to underlying instances of the main ''daemon'' process, bog down or stop your machine with too many instances of itself, open so many file descriptors that you are prevented even from rebooting the machine, serve as a proxy for the malicious use of other programs, and so on. Daemons aren't usually evil; they're just misunderstood. Be aware of the [[:Category:Securing FreeBSD|security implications]] of your configuration and startup options.

[[Category:FreeBSD Terminology]]

Rc.subr

2006-06-20T20:41:23Z

Ninereasons:

'''rc.subr''' is a subroutine library that was developed primarily for the NetBSD system and imported into FreeBSD where it was further developed. See [http://www.freebsd.org/cgi/man.cgi?query=rc.subr&apropos=0&sektion=0&manpath=FreeBSD+6.0-RELEASE+and+Ports&format=html man rc.subr]

Under FreeBSD 4.x, you may have noticed this file being installed into ''/usr/local/etc/rc.subr'', where it was increasingly used by port-installed daemons. Beginning in FreeBSD 5.0, this file became integral to the FreeBSD startup mechanism for both, base-installed services and port-installed services. At the same time, the directory ''/etc/rc.d'' appeared in FreeBSD, where you will find startup scripts for all base-installed services (so that, for example, ''/etc/rc.sendmail'' disappeared, and was replaced by ''/[[etc/rc.d/sendmail]]'', which uses the ''rc.subr'' mechanism)

Besides being a repository of some very useful and well-written functions, perhaps the chief advantage of using the file is that it ties the configuration, startup and shutdown of these daemons to /[[etc/rc.conf]] in a consistent way. For example, unless the service has an <code>="YES"</code> line in ''rc.conf'', the ''start'' and ''stop'' commands have no effect.

[[User:Jimbo|Some folks]] don't like starting all of their services from ''/etc/rc.conf''. ''' ''rc.subr'' ''' accommodates this preference with the prefixes, ''force'' and ''one''.

Another significant change that this mechanism introduces is, if a startup script ends with ''.sh'', it will be loaded into the current shell (instead of a subshell); ''' ''N.B.'' ''': this means that ''if ''script.sh'' fails'', the remainder of the startup sequence will not be executed.

Finally, a valuable improvement to service startup is better control over the order in which [[daemon]]s are started. See [http://www.freebsd.org/cgi/man.cgi?query=rc&sektion=8&apropos=0&manpath=FreeBSD+6.0-RELEASE+and+Ports man rc(8)] and [http://www.freebsd.org/cgi/man.cgi?query=rcorder&sektion=8&apropos=0&manpath=FreeBSD+6.0-stable man rcorder(8)]

===Examples===
1. Minimal ''rc.d/'' script example, using the new ''rc.subr'' mechanism. Most scripts require little more than this.
#!/bin/sh
#
# PROVIDE: foo
# REQUIRE: bar_service_required_to_precede_foo
# BEFORE: baz_service_requiring_foo_to_precede_it

. /etc/rc.subr

name="foo"
rcvar=`set_rcvar`
command="/usr/local/bin/foo"
load_rc_config $name
run_rc_command "$1"

2. Manually start and stop mysql, even though <code>mysql_enable</code> is set to <code>"NO"</code> (default) in ''rc.conf''.
* Skip all failing pre-requisite "''required_*''" tests (if any).
# /usr/local/etc/rc.d/mysql-server forcestart
# /usr/local/etc/rc.d/mysql-server forcestop
* Skip only the <code>="YES"</code> test
# /usr/local/etc/rc.d/mysql-server onestart
# /usr/local/etc/rc.d/mysql-server onestop

3. List the sequence in which all services will be started, reporting missing dependencies:
# rcorder /etc/rc.d/* /usr/local/etc/rc.d/*

[[Category:Important Config Files]]

Usr/local/etc/rc.d

2006-06-20T20:40:51Z

Ninereasons: Moving section to a separate article

'''/usr/local/etc/rc.d''' isn't actually a config file, it's a directory. Any executable shell script in this directory will be executed (using the [[Bourne shell]]) with the argument "start" when the system boots, and again with the argument "stop" when the system is shut down.

You may find this generic rc.d script (shown as I use it to start bind9) handy for general purpose use:

#!/bin/sh

case "$1" in
start)
/usr/sbin/named -c /etc/namedb/named.conf &
echo "bind9"
exit 0
;;
stop)
exec killall named
;;
*)
echo "Usage: `basename $0` {start|stop}" >&2
exit 64
;;
esac

Note that scripts in /usr/local/etc/rc.d cannot and will not execute, at boot time or otherwise, if you do not set the permissions to allow their execution. In most cases, you will want to [[chmod]] 755 any rc.d scripts, though you may also consider 700 on root-owned rc.d scripts to make sure that unprivileged users don't mess with attempting to start or stop sensitive services.

The most recent versions of FreeBSD make [[rc.conf]] more integral to the startup of local services. For information about the way that [[daemon]]s are started in the newest versions of FreeBSD, see [[rc.subr]].

[[Category:Important Config Files]]