1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
1776
1777
1778
1779
1780
1781
1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
1904
1905
1906
1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
2048
2049
2050
2051
2052
2053
2054
2055
2056
2057
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
2108
2109
2110
2111
2112
2113
2114
2115
2116
2117
2118
2119
2120
2121
2122
2123
2124
2125
2126
2127
2128
2129
2130
2131
2132
2133
2134
2135
2136
2137
2138
2139
2140
2141
2142
2143
2144
2145
2146
2147
2148
2149
2150
2151
2152
2153
2154
2155
2156
2157
2158
2159
2160
2161
2162
2163
2164
2165
2166
2167
2168
2169
2170
2171
2172
2173
2174
2175
2176
2177
2178
2179
2180
2181
2182
2183
2184
2185
2186
2187
2188
2189
2190
2191
2192
2193
2194
2195
2196
2197
2198
2199
2200
2201
2202
2203
2204
2205
2206
2207
2208
2209
2210
2211
2212
2213
2214
2215
2216
2217
2218
2219
2220
2221
2222
2223
2224
2225
2226
2227
2228
2229
2230
2231
2232
2233
2234
2235
2236
2237
2238
2239
2240
2241
2242
2243
2244
2245
2246
2247
2248
2249
2250
2251
2252
2253
2254
2255
2256
2257
2258
2259
2260
2261
2262
2263
2264
2265
2266
2267
2268
2269
2270
2271
2272
2273
2274
2275
2276
2277
2278
2279
2280
2281
2282
2283
2284
2285
2286
2287
2288
2289
2290
2291
2292
2293
2294
2295
2296
2297
2298
2299
2300
2301
2302
2303
2304
2305
2306
2307
2308
2309
2310
2311
2312
2313
2314
2315
2316
2317
2318
2319
2320
2321
2322
2323
2324
2325
2326
2327
2328
2329
2330
2331
2332
2333
2334
2335
2336
2337
2338
2339
2340
2341
2342
2343
2344
2345
2346
2347
2348
2349
2350
2351
2352
2353
2354
2355
2356
2357
2358
2359
2360
2361
2362
2363
2364
2365
|
<!-- doc/src/sgml/runtime.sgml -->
<chapter id="runtime">
<title>Server Setup and Operation</title>
<para>
This chapter discusses how to set up and run the database server
and its interactions with the operating system.
</para>
<sect1 id="postgres-user">
<title>The <productname>PostgreSQL</productname> User Account</title>
<indexterm>
<primary>postgres user</primary>
</indexterm>
<para>
As with any server daemon that is accessible to the outside world,
it is advisable to run <productname>PostgreSQL</productname> under a
separate user account. This user account should only own the data
that is managed by the server, and should not be shared with other
daemons. (For example, using the user <literal>nobody</literal> is a bad
idea.) It is not advisable to install executables owned by this
user because compromised systems could then modify their own
binaries.
</para>
<para>
To add a Unix user account to your system, look for a command
<command>useradd</command> or <command>adduser</command>. The user
name <systemitem>postgres</systemitem> is often used, and is assumed
throughout this book, but you can use another name if you like.
</para>
</sect1>
<sect1 id="creating-cluster">
<title>Creating a Database Cluster</title>
<indexterm>
<primary>database cluster</primary>
</indexterm>
<indexterm>
<primary>data area</primary>
<see>database cluster</see>
</indexterm>
<para>
Before you can do anything, you must initialize a database storage
area on disk. We call this a <firstterm>database cluster</firstterm>.
(<acronym>SQL</acronym> uses the term catalog cluster.) A
database cluster is a collection of databases that is managed by a
single instance of a running database server. After initialization, a
database cluster will contain a database named <literal>postgres</literal>,
which is meant as a default database for use by utilities, users and third
party applications. The database server itself does not require the
<literal>postgres</literal> database to exist, but many external utility
programs assume it exists. Another database created within each cluster
during initialization is called
<literal>template1</literal>. As the name suggests, this will be used
as a template for subsequently created databases; it should not be
used for actual work. (See <xref linkend="managing-databases"> for
information about creating new databases within a cluster.)
</para>
<para>
In file system terms, a database cluster will be a single directory
under which all data will be stored. We call this the <firstterm>data
directory</firstterm> or <firstterm>data area</firstterm>. It is
completely up to you where you choose to store your data. There is no
default, although locations such as
<filename>/usr/local/pgsql/data</filename> or
<filename>/var/lib/pgsql/data</filename> are popular. To initialize a
database cluster, use the command <xref
linkend="app-initdb">,<indexterm><primary>initdb</></> which is
installed with <productname>PostgreSQL</productname>. The desired
file system location of your database cluster is indicated by the
<option>-D</option> option, for example:
<screen>
<prompt>$</> <userinput>initdb -D /usr/local/pgsql/data</userinput>
</screen>
Note that you must execute this command while logged into the
<productname>PostgreSQL</productname> user account, which is
described in the previous section.
</para>
<tip>
<para>
As an alternative to the <option>-D</option> option, you can set
the environment variable <envar>PGDATA</envar>.
<indexterm><primary><envar>PGDATA</envar></primary></indexterm>
</para>
</tip>
<para>
Alternatively, you can run <command>initdb</command> via
the <xref linkend="app-pg-ctl">
program<indexterm><primary>pg_ctl</></> like so:
<screen>
<prompt>$</> <userinput>pg_ctl -D /usr/local/pgsql/data initdb</userinput>
</screen>
This may be more intuitive if you are
using <command>pg_ctl</command> for starting and stopping the
server (see <xref linkend="server-start">), so
that <command>pg_ctl</command> would be the sole command you use
for managing the database server instance.
</para>
<para>
<command>initdb</command> will attempt to create the directory you
specify if it does not already exist. It is likely that it will not
have the permission to do so (if you followed our advice and created
an unprivileged account). In that case you should create the
directory yourself (as root) and change the owner to be the
<productname>PostgreSQL</productname> user. Here is how this might
be done:
<screen>
root# <userinput>mkdir /usr/local/pgsql/data</userinput>
root# <userinput>chown postgres /usr/local/pgsql/data</userinput>
root# <userinput>su postgres</userinput>
postgres$ <userinput>initdb -D /usr/local/pgsql/data</userinput>
</screen>
</para>
<para>
<command>initdb</command> will refuse to run if the data directory
looks like it has already been initialized.</para>
<para>
Because the data directory contains all the data stored in the
database, it is essential that it be secured from unauthorized
access. <command>initdb</command> therefore revokes access
permissions from everyone but the
<productname>PostgreSQL</productname> user.
</para>
<para>
However, while the directory contents are secure, the default
client authentication setup allows any local user to connect to the
database and even become the database superuser. If you do not
trust other local users, we recommend you use one of
<command>initdb</command>'s <option>-W</option>, <option>--pwprompt</option>
or <option>--pwfile</option> options to assign a password to the
database superuser.<indexterm>
<primary>password</>
<secondary>of the superuser</>
</indexterm>
Also, specify <option>-A md5</> or
<option>-A password</> so that the default <literal>trust</> authentication
mode is not used; or modify the generated <filename>pg_hba.conf</filename>
file after running <command>initdb</command>, but
<emphasis>before</> you start the server for the first time. (Other
reasonable approaches include using <literal>peer</literal> authentication
or file system permissions to restrict connections. See <xref
linkend="client-authentication"> for more information.)
</para>
<para>
<command>initdb</command> also initializes the default
locale<indexterm><primary>locale</></> for the database cluster.
Normally, it will just take the locale settings in the environment
and apply them to the initialized database. It is possible to
specify a different locale for the database; more information about
that can be found in <xref linkend="locale">. The default sort order used
within the particular database cluster is set by
<command>initdb</command>, and while you can create new databases using
different sort order, the order used in the template databases that initdb
creates cannot be changed without dropping and recreating them.
There is also a performance impact for using locales
other than <literal>C</> or <literal>POSIX</>. Therefore, it is
important to make this choice correctly the first time.
</para>
<para>
<command>initdb</command> also sets the default character set encoding
for the database cluster. Normally this should be chosen to match the
locale setting. For details see <xref linkend="multibyte">.
</para>
<sect2 id="creating-cluster-nfs">
<title>Network File Systems</title>
<indexterm zone="creating-cluster-nfs">
<primary>Network File Systems</primary>
</indexterm>
<indexterm><primary><acronym>NFS</></><see>Network File Systems</></>
<indexterm><primary>Network Attached Storage (<acronym>NAS</>)</><see>Network File Systems</></>
<para>
Many installations create database clusters on network file systems.
Sometimes this is done directly via <acronym>NFS</>, or by using a
Network Attached Storage (<acronym>NAS</>) device that uses
<acronym>NFS</> internally. <productname>PostgreSQL</> does nothing
special for <acronym>NFS</> file systems, meaning it assumes
<acronym>NFS</> behaves exactly like locally-connected drives
(<acronym>DAS</>, Direct Attached Storage). If client and server
<acronym>NFS</> implementations have non-standard semantics, this can
cause reliability problems (see <ulink
url="http://www.time-travellers.org/shane/papers/NFS_considered_harmful.html"></ulink>).
Specifically, delayed (asynchronous) writes to the <acronym>NFS</>
server can cause reliability problems; if possible, mount
<acronym>NFS</> file systems synchronously (without caching) to avoid
this. Also, soft-mounting <acronym>NFS</> is not recommended.
(Storage Area Networks (<acronym>SAN</>) use a low-level
communication protocol rather than <acronym>NFS</>.)
</para>
</sect2>
</sect1>
<sect1 id="server-start">
<title>Starting the Database Server</title>
<para>
Before anyone can access the database, you must start the database
server. The database server program is called
<command>postgres</command>.<indexterm><primary>postgres</></>
The <command>postgres</command> program must know where to
find the data it is supposed to use. This is done with the
<option>-D</option> option. Thus, the simplest way to start the
server is:
<screen>
$ <userinput>postgres -D /usr/local/pgsql/data</userinput>
</screen>
which will leave the server running in the foreground. This must be
done while logged into the <productname>PostgreSQL</productname> user
account. Without <option>-D</option>, the server will try to use
the data directory named by the environment variable <envar>PGDATA</envar>.
If that variable is not provided either, it will fail.
</para>
<para>
Normally it is better to start <command>postgres</command> in the
background. For this, use the usual Unix shell syntax:
<screen>
$ <userinput>postgres -D /usr/local/pgsql/data >logfile 2>&1 &</userinput>
</screen>
It is important to store the server's <systemitem>stdout</> and
<systemitem>stderr</> output somewhere, as shown above. It will help
for auditing purposes and to diagnose problems. (See <xref
linkend="logfile-maintenance"> for a more thorough discussion of log
file handling.)
</para>
<para>
The <command>postgres</command> program also takes a number of other
command-line options. For more information, see the
<xref linkend="app-postgres"> reference page
and <xref linkend="runtime-config"> below.
</para>
<para>
This shell syntax can get tedious quickly. Therefore the wrapper
program
<xref linkend="app-pg-ctl"><indexterm><primary>pg_ctl</primary></indexterm>
is provided to simplify some tasks. For example:
<programlisting>
pg_ctl start -l logfile
</programlisting>
will start the server in the background and put the output into the
named log file. The <option>-D</option> option has the same meaning
here as for <command>postgres</command>. <command>pg_ctl</command>
is also capable of stopping the server.
</para>
<para>
Normally, you will want to start the database server when the
computer boots.<indexterm>
<primary>booting</>
<secondary>starting the server during</>
</indexterm>
Autostart scripts are operating-system-specific.
There are a few distributed with
<productname>PostgreSQL</productname> in the
<filename>contrib/start-scripts</> directory. Installing one will require
root privileges.
</para>
<para>
Different systems have different conventions for starting up daemons
at boot time. Many systems have a file
<filename>/etc/rc.local</filename> or
<filename>/etc/rc.d/rc.local</filename>. Others use <filename>init.d</filename> or
<filename>rc.d</> directories. Whatever you do, the server must be
run by the <productname>PostgreSQL</productname> user account
<emphasis>and not by root</emphasis> or any other user. Therefore you
probably should form your commands using
<literal>su postgres -c '...'</literal>. For example:
<programlisting>
su postgres -c 'pg_ctl start -D /usr/local/pgsql/data -l serverlog'
</programlisting>
</para>
<para>
Here are a few more operating-system-specific suggestions. (In each
case be sure to use the proper installation directory and user
name where we show generic values.)
<itemizedlist>
<listitem>
<para>
For <productname>FreeBSD</productname>, look at the file
<filename>contrib/start-scripts/freebsd</filename> in the
<productname>PostgreSQL</productname> source distribution.
<indexterm><primary>FreeBSD</><secondary>start script</secondary></>
</para>
</listitem>
<listitem>
<para>
On <productname>OpenBSD</productname>, add the following lines
to the file <filename>/etc/rc.local</filename>:
<indexterm><primary>OpenBSD</><secondary>start script</secondary></>
<programlisting>
if [ -x /usr/local/pgsql/bin/pg_ctl -a -x /usr/local/pgsql/bin/postgres ]; then
su -l postgres -c '/usr/local/pgsql/bin/pg_ctl start -s -l /var/postgresql/log -D /usr/local/pgsql/data'
echo -n ' postgresql'
fi
</programlisting>
</para>
</listitem>
<listitem>
<para>
On <productname>Linux</productname> systems either add
<indexterm><primary>Linux</><secondary>start script</secondary></>
<programlisting>
/usr/local/pgsql/bin/pg_ctl start -l logfile -D /usr/local/pgsql/data
</programlisting>
to <filename>/etc/rc.d/rc.local</filename>
or <filename>/etc/rc.local</filename> or look at the file
<filename>contrib/start-scripts/linux</filename> in the
<productname>PostgreSQL</productname> source distribution.
</para>
</listitem>
<listitem>
<para>
On <productname>NetBSD</productname>, use either the
<productname>FreeBSD</productname> or
<productname>Linux</productname> start scripts, depending on
preference.
<indexterm><primary>NetBSD</><secondary>start script</secondary></>
</para>
</listitem>
<listitem>
<para>
On <productname>Solaris</productname>, create a file called
<filename>/etc/init.d/postgresql</filename> that contains
the following line:
<indexterm><primary>Solaris</><secondary>start script</secondary></>
<programlisting>
su - postgres -c "/usr/local/pgsql/bin/pg_ctl start -l logfile -D /usr/local/pgsql/data"
</programlisting>
Then, create a symbolic link to it in <filename>/etc/rc3.d</> as
<filename>S99postgresql</>.
</para>
</listitem>
</itemizedlist>
</para>
<para>
While the server is running, its
<acronym>PID</acronym> is stored in the file
<filename>postmaster.pid</filename> in the data directory. This is
used to prevent multiple server instances from
running in the same data directory and can also be used for
shutting down the server.
</para>
<sect2 id="server-start-failures">
<title>Server Start-up Failures</title>
<para>
There are several common reasons the server might fail to
start. Check the server's log file, or start it by hand (without
redirecting standard output or standard error) and see what error
messages appear. Below we explain some of the most common error
messages in more detail.
</para>
<para>
<screen>
LOG: could not bind IPv4 socket: Address already in use
HINT: Is another postmaster already running on port 5432? If not, wait a few seconds and retry.
FATAL: could not create TCP/IP listen socket
</screen>
This usually means just what it suggests: you tried to start
another server on the same port where one is already running.
However, if the kernel error message is not <computeroutput>Address
already in use</computeroutput> or some variant of that, there might
be a different problem. For example, trying to start a server
on a reserved port number might draw something like:
<screen>
$ <userinput>postgres -p 666</userinput>
LOG: could not bind IPv4 socket: Permission denied
HINT: Is another postmaster already running on port 666? If not, wait a few seconds and retry.
FATAL: could not create TCP/IP listen socket
</screen>
</para>
<para>
A message like:
<screen>
FATAL: could not create shared memory segment: Invalid argument
DETAIL: Failed system call was shmget(key=5440001, size=4011376640, 03600).
</screen>
probably means your kernel's limit on the size of shared memory is
smaller than the work area <productname>PostgreSQL</productname>
is trying to create (4011376640 bytes in this example). Or it could
mean that you do not have System-V-style shared memory support
configured into your kernel at all. As a temporary workaround, you
can try starting the server with a smaller-than-normal number of
buffers (<xref linkend="guc-shared-buffers">). You will eventually want
to reconfigure your kernel to increase the allowed shared memory
size. You might also see this message when trying to start multiple
servers on the same machine, if their total space requested
exceeds the kernel limit.
</para>
<para>
An error like:
<screen>
FATAL: could not create semaphores: No space left on device
DETAIL: Failed system call was semget(5440126, 17, 03600).
</screen>
does <emphasis>not</emphasis> mean you've run out of disk
space. It means your kernel's limit on the number of <systemitem
class="osname">System V</> semaphores is smaller than the number
<productname>PostgreSQL</productname> wants to create. As above,
you might be able to work around the problem by starting the
server with a reduced number of allowed connections
(<xref linkend="guc-max-connections">), but you'll eventually want to
increase the kernel limit.
</para>
<para>
If you get an <quote>illegal system call</> error, it is likely that
shared memory or semaphores are not supported in your kernel at
all. In that case your only option is to reconfigure the kernel to
enable these features.
</para>
<para>
Details about configuring <systemitem class="osname">System V</>
<acronym>IPC</> facilities are given in <xref linkend="sysvipc">.
</para>
</sect2>
<sect2 id="client-connection-problems">
<title>Client Connection Problems</title>
<para>
Although the error conditions possible on the client side are quite
varied and application-dependent, a few of them might be directly
related to how the server was started. Conditions other than
those shown below should be documented with the respective client
application.
</para>
<para>
<screen>
psql: could not connect to server: Connection refused
Is the server running on host "server.joe.com" and accepting
TCP/IP connections on port 5432?
</screen>
This is the generic <quote>I couldn't find a server to talk
to</quote> failure. It looks like the above when TCP/IP
communication is attempted. A common mistake is to forget to
configure the server to allow TCP/IP connections.
</para>
<para>
Alternatively, you'll get this when attempting Unix-domain socket
communication to a local server:
<screen>
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
</screen>
</para>
<para>
The last line is useful in verifying that the client is trying to
connect to the right place. If there is in fact no server
running there, the kernel error message will typically be either
<computeroutput>Connection refused</computeroutput> or
<computeroutput>No such file or directory</computeroutput>, as
illustrated. (It is important to realize that
<computeroutput>Connection refused</computeroutput> in this context
does <emphasis>not</emphasis> mean that the server got your
connection request and rejected it. That case will produce a
different message, as shown in <xref
linkend="client-authentication-problems">.) Other error messages
such as <computeroutput>Connection timed out</computeroutput> might
indicate more fundamental problems, like lack of network
connectivity.
</para>
</sect2>
</sect1>
<sect1 id="kernel-resources">
<title>Managing Kernel Resources</title>
<para>
<productname>PostgreSQL</> can sometimes exhaust various operating system
resource limits, especially when multiple copies of the server are running
on the same system, or in very large installations. This section explains
the kernel resources used by <productname>PostgreSQL</> and the steps you
can take to resolve problems related to kernel resource consumption.
</para>
<sect2 id="sysvipc">
<title>Shared Memory and Semaphores</title>
<indexterm zone="sysvipc">
<primary>shared memory</primary>
</indexterm>
<indexterm zone="sysvipc">
<primary>semaphores</primary>
</indexterm>
<para>
Shared memory and semaphores are collectively referred to as
<quote><systemitem class="osname">System V</>
<acronym>IPC</></quote> (together with message queues, which are not
relevant for <productname>PostgreSQL</>). Except on
<systemitem class="osname">Windows</>, where <productname>PostgreSQL</>
provides its own replacement implementation of these facilities, these
facilities are required in order to run
<productname>PostgreSQL</>.
</para>
<para>
The complete lack of these facilities is usually manifested by an
<errorname>Illegal system call</> error upon server start. In
that case there is no alternative but to reconfigure your
kernel. <productname>PostgreSQL</> won't work without them.
This situation is rare, however, among modern operating systems.
</para>
<para>
When <productname>PostgreSQL</> exceeds one of the various hard
<acronym>IPC</> limits, the server will refuse to start and
should leave an instructive error message describing the problem
and what to do about it. (See also <xref
linkend="server-start-failures">.) The relevant kernel
parameters are named consistently across different systems; <xref
linkend="sysvipc-parameters"> gives an overview. The methods to set
them, however, vary. Suggestions for some platforms are given below.
</para>
<note>
<para>
Prior to <productname>PostgreSQL</> 9.3, the amount of System V shared
memory required to start the server was much larger. If you are running
an older version of the server, please consult the documentation for
your server version.
</para>
</note>
<table id="sysvipc-parameters">
<title><systemitem class="osname">System V</> <acronym>IPC</> Parameters</title>
<tgroup cols="3">
<thead>
<row>
<entry>Name</>
<entry>Description</>
<entry>Reasonable values</>
</row>
</thead>
<tbody>
<row>
<entry><varname>SHMMAX</></>
<entry>Maximum size of shared memory segment (bytes)</>
<entry>at least 1kB (more if running many copies of the server)</entry>
</row>
<row>
<entry><varname>SHMMIN</></>
<entry>Minimum size of shared memory segment (bytes)</>
<entry>1</>
</row>
<row>
<entry><varname>SHMALL</></>
<entry>Total amount of shared memory available (bytes or pages)</>
<entry>if bytes, same as <varname>SHMMAX</varname>; if pages, <literal>ceil(SHMMAX/PAGE_SIZE)</literal></>
</row>
<row>
<entry><varname>SHMSEG</></>
<entry>Maximum number of shared memory segments per process</>
<entry>only 1 segment is needed, but the default is much higher</>
</row>
<row>
<entry><varname>SHMMNI</></>
<entry>Maximum number of shared memory segments system-wide</>
<entry>like <varname>SHMSEG</> plus room for other applications</>
</row>
<row>
<entry><varname>SEMMNI</></>
<entry>Maximum number of semaphore identifiers (i.e., sets)</>
<entry>at least <literal>ceil((max_connections + autovacuum_max_workers + 4) / 16)</literal></>
</row>
<row>
<entry><varname>SEMMNS</></>
<entry>Maximum number of semaphores system-wide</>
<entry><literal>ceil((max_connections + autovacuum_max_workers + 4) / 16) * 17</literal> plus room for other applications</>
</row>
<row>
<entry><varname>SEMMSL</></>
<entry>Maximum number of semaphores per set</>
<entry>at least 17</>
</row>
<row>
<entry><varname>SEMMAP</></>
<entry>Number of entries in semaphore map</>
<entry>see text</>
</row>
<row>
<entry><varname>SEMVMX</></>
<entry>Maximum value of semaphore</>
<entry>at least 1000 (The default is often 32767; do not change unless necessary)</>
</row>
</tbody>
</tgroup>
</table>
<para>
<productname>PostgreSQL</> requires a few bytes of System V shared memory
(typically 48 bytes, on 64-bit platforms) for each copy of the server.
On most modern operating systems, this amount can easily be allocated.
However, if you are running many copies of the server, or if other
applications are also using System V shared memory, it may be necessary
to increase <varname>SHMMAX</>, the maximum size in bytes of a shared
memory segment, or <varname>SHMALL</>, the total amount of System V shared
memory system-wide. Note that <varname>SHMALL</> is measured in pages
rather than bytes on many systems.
</para>
<para>
Less likely to cause problems is the minimum size for shared
memory segments (<varname>SHMMIN</>), which should be at most
approximately 32 bytes for <productname>PostgreSQL</> (it is
usually just 1). The maximum number of segments system-wide
(<varname>SHMMNI</>) or per-process (<varname>SHMSEG</>) are unlikely
to cause a problem unless your system has them set to zero.
</para>
<para>
<productname>PostgreSQL</> uses one semaphore per allowed connection
(<xref linkend="guc-max-connections">) and allowed autovacuum worker
process (<xref linkend="guc-autovacuum-max-workers">), in sets of 16.
Each such set will
also contain a 17th semaphore which contains a <quote>magic
number</quote>, to detect collision with semaphore sets used by
other applications. The maximum number of semaphores in the system
is set by <varname>SEMMNS</>, which consequently must be at least
as high as <varname>max_connections</> plus
<varname>autovacuum_max_workers</>, plus one extra for each 16
allowed connections plus workers (see the formula in <xref
linkend="sysvipc-parameters">). The parameter <varname>SEMMNI</>
determines the limit on the number of semaphore sets that can
exist on the system at one time. Hence this parameter must be at
least <literal>ceil((max_connections + autovacuum_max_workers + 4) / 16)</>.
Lowering the number
of allowed connections is a temporary workaround for failures,
which are usually confusingly worded <quote>No space
left on device</>, from the function <function>semget</>.
</para>
<para>
In some cases it might also be necessary to increase
<varname>SEMMAP</> to be at least on the order of
<varname>SEMMNS</>. This parameter defines the size of the semaphore
resource map, in which each contiguous block of available semaphores
needs an entry. When a semaphore set is freed it is either added to
an existing entry that is adjacent to the freed block or it is
registered under a new map entry. If the map is full, the freed
semaphores get lost (until reboot). Fragmentation of the semaphore
space could over time lead to fewer available semaphores than there
should be.
</para>
<para>
The <varname>SEMMSL</> parameter, which determines how many
semaphores can be in a set, must be at least 17 for
<productname>PostgreSQL</>.
</para>
<para>
Various other settings related to <quote>semaphore undo</>, such as
<varname>SEMMNU</> and <varname>SEMUME</>, do not affect
<productname>PostgreSQL</>.
</para>
<variablelist>
<varlistentry>
<term><systemitem class="osname">AIX</>
<indexterm><primary>AIX</><secondary>IPC configuration</></>
</term>
<listitem>
<para>
At least as of version 5.1, it should not be necessary to do
any special configuration for such parameters as
<varname>SHMMAX</varname>, as it appears this is configured to
allow all memory to be used as shared memory. That is the
sort of configuration commonly used for other databases such
as <application>DB/2</application>.</para>
<para> It might, however, be necessary to modify the global
<command>ulimit</command> information in
<filename>/etc/security/limits</filename>, as the default hard
limits for file sizes (<varname>fsize</varname>) and numbers of
files (<varname>nofiles</varname>) might be too low.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><systemitem class="osname">FreeBSD</>
<indexterm><primary>FreeBSD</><secondary>IPC configuration</></>
</term>
<listitem>
<para>
The default settings can be changed using
the <command>sysctl</command> or
<command>loader</command> interfaces. The following
parameters can be set using <command>sysctl</command>:
<screen>
<prompt>#</prompt> <userinput>sysctl kern.ipc.shmall=32768</userinput>
<prompt>#</prompt> <userinput>sysctl kern.ipc.shmmax=134217728</userinput>
</screen>
To make these settings persist over reboots, modify
<filename>/etc/sysctl.conf</filename>.
</para>
<para>
These semaphore-related settings are read-only as far as
<command>sysctl</command> is concerned, but can be set in
<filename>/boot/loader.conf</filename>:
<programlisting>
kern.ipc.semmni=256
kern.ipc.semmns=512
kern.ipc.semmnu=256
</programlisting>
After modifying these values a reboot is required for the new
settings to take effect.
(Note: FreeBSD does not use <varname>SEMMAP</>. Older versions
would accept but ignore a setting for <literal>kern.ipc.semmap</>;
newer versions reject it altogether.)
</para>
<para>
You might also want to configure your kernel to lock shared
memory into RAM and prevent it from being paged out to swap.
This can be accomplished using the <command>sysctl</command>
setting <literal>kern.ipc.shm_use_phys</literal>.
</para>
<para>
If running in FreeBSD jails by enabling <application>sysctl</>'s
<literal>security.jail.sysvipc_allowed</>, <application>postmaster</>s
running in different jails should be run by different operating system
users. This improves security because it prevents non-root users
from interfering with shared memory or semaphores in different jails,
and it allows the PostgreSQL IPC cleanup code to function properly.
(In FreeBSD 6.0 and later the IPC cleanup code does not properly detect
processes in other jails, preventing the running of postmasters on the
same port in different jails.)
</para>
<para>
<systemitem class="osname">FreeBSD</> versions before 4.0 work like
<systemitem class="osname">OpenBSD</> (see below).
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><systemitem class="osname">NetBSD</>
<indexterm><primary>NetBSD</><secondary>IPC configuration</></>
</term>
<listitem>
<para>
In <systemitem class="osname">NetBSD</> 5.0 and later,
IPC parameters can be adjusted using <command>sysctl</command>,
for example:
<screen>
<prompt>$</prompt> <userinput>sysctl -w kern.ipc.shmmax=16777216</userinput>
</screen>
To have these settings persist over reboots, modify
<filename>/etc/sysctl.conf</filename>.
</para>
<para>
You might also want to configure your kernel to lock shared
memory into RAM and prevent it from being paged out to swap.
This can be accomplished using the <command>sysctl</command>
setting <literal>kern.ipc.shm_use_phys</literal>.
</para>
<para>
<systemitem class="osname">NetBSD</> versions before 5.0 work like
<systemitem class="osname">OpenBSD</> (see below), except that
parameters should be set with the keyword <literal>options</> not
<literal>option</>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><systemitem class="osname">OpenBSD</>
<indexterm><primary>OpenBSD</><secondary>IPC configuration</></>
</term>
<listitem>
<para>
The options <varname>SYSVSHM</> and <varname>SYSVSEM</> need
to be enabled when the kernel is compiled. (They are by
default.) The maximum size of shared memory is determined by
the option <varname>SHMMAXPGS</> (in pages). The following
shows an example of how to set the various parameters:
<programlisting>
option SYSVSHM
option SHMMAXPGS=4096
option SHMSEG=256
option SYSVSEM
option SEMMNI=256
option SEMMNS=512
option SEMMNU=256
option SEMMAP=256
</programlisting>
</para>
<para>
You might also want to configure your kernel to lock shared
memory into RAM and prevent it from being paged out to swap.
This can be accomplished using the <command>sysctl</command>
setting <literal>kern.ipc.shm_use_phys</literal>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><systemitem class="osname">HP-UX</>
<indexterm><primary>HP-UX</><secondary>IPC configuration</></>
</term>
<listitem>
<para>
The default settings tend to suffice for normal installations.
On <productname>HP-UX</> 10, the factory default for
<varname>SEMMNS</> is 128, which might be too low for larger
database sites.
</para>
<para>
<acronym>IPC</> parameters can be set in the <application>System
Administration Manager</> (<acronym>SAM</>) under
<menuchoice><guimenu>Kernel
Configuration</><guimenuitem>Configurable Parameters</></>. Choose
<guibutton>Create A New Kernel</> when you're done.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><systemitem class="osname">Linux</>
<indexterm><primary>Linux</><secondary>IPC configuration</></>
</term>
<listitem>
<para>
The default maximum segment size is 32 MB, and the
default maximum total size is 2097152
pages. A page is almost always 4096 bytes except in unusual
kernel configurations with <quote>huge pages</quote>
(use <literal>getconf PAGE_SIZE</literal> to verify).
</para>
<para>
The shared memory size settings can be changed via the
<command>sysctl</command> interface. For example, to allow 16 GB:
<screen>
<prompt>$</prompt> <userinput>sysctl -w kernel.shmmax=17179869184</userinput>
<prompt>$</prompt> <userinput>sysctl -w kernel.shmall=4194304</userinput>
</screen>
In addition these settings can be preserved between reboots in
the file <filename>/etc/sysctl.conf</filename>. Doing that is
highly recommended.
</para>
<para>
Ancient distributions might not have the <command>sysctl</command> program,
but equivalent changes can be made by manipulating the
<filename>/proc</filename> file system:
<screen>
<prompt>$</prompt> <userinput>echo 17179869184 >/proc/sys/kernel/shmmax</userinput>
<prompt>$</prompt> <userinput>echo 4194304 >/proc/sys/kernel/shmall</userinput>
</screen>
</para>
<para>
The remaining defaults are quite generously sized, and usually
do not require changes.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><systemitem class="osname">Mac OS X</>
<indexterm><primary>Mac OS X</><secondary>IPC configuration</></>
</term>
<listitem>
<para>
The recommended method for configuring shared memory in OS X
is to create a file named <filename>/etc/sysctl.conf</>,
containing variable assignments such as:
<programlisting>
kern.sysv.shmmax=4194304
kern.sysv.shmmin=1
kern.sysv.shmmni=32
kern.sysv.shmseg=8
kern.sysv.shmall=1024
</programlisting>
Note that in some OS X versions,
<emphasis>all five</> shared-memory parameters must be set in
<filename>/etc/sysctl.conf</>, else the values will be ignored.
</para>
<para>
Beware that recent releases of OS X ignore attempts to set
<varname>SHMMAX</> to a value that isn't an exact multiple of 4096.
</para>
<para>
<varname>SHMALL</> is measured in 4 kB pages on this platform.
</para>
<para>
In older OS X versions, you will need to reboot to have changes in the
shared memory parameters take effect. As of 10.5 it is possible to
change all but <varname>SHMMNI</> on the fly, using
<application>sysctl</>. But it's still best to set up your preferred
values via <filename>/etc/sysctl.conf</>, so that the values will be
kept across reboots.
</para>
<para>
The file <filename>/etc/sysctl.conf</> is only honored in OS X
10.3.9 and later. If you are running a previous 10.3.x release,
you must edit the file <filename>/etc/rc</>
and change the values in the following commands:
<programlisting>
sysctl -w kern.sysv.shmmax
sysctl -w kern.sysv.shmmin
sysctl -w kern.sysv.shmmni
sysctl -w kern.sysv.shmseg
sysctl -w kern.sysv.shmall
</programlisting>
Note that
<filename>/etc/rc</> is usually overwritten by OS X system updates,
so you should expect to have to redo these edits after each update.
</para>
<para>
In OS X 10.2 and earlier, instead edit these commands in the file
<filename>/System/Library/StartupItems/SystemTuning/SystemTuning</>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><systemitem class="osname">SCO OpenServer</>
<indexterm><primary>SCO OpenServer</><secondary>IPC configuration</></>
</term>
<listitem>
<para>
In the default configuration, only 512 kB of shared memory per
segment is allowed. To increase the setting, first change to the
directory <filename>/etc/conf/cf.d</>. To display the current value of
<varname>SHMMAX</>, run:
<programlisting>
./configure -y SHMMAX
</programlisting>
To set a new value for <varname>SHMMAX</>, run:
<programlisting>
./configure SHMMAX=<replaceable>value</>
</programlisting>
where <replaceable>value</> is the new value you want to use
(in bytes). After setting <varname>SHMMAX</>, rebuild the kernel:
<programlisting>
./link_unix
</programlisting>
and reboot.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><systemitem class="osname">Solaris</> 2.6 to 2.9 (Solaris
6 to Solaris 9)
<indexterm><primary>Solaris</><secondary>IPC configuration</></>
</term>
<listitem>
<para>
The relevant settings can be changed in
<filename>/etc/system</>, for example:
<programlisting>
set shmsys:shminfo_shmmax=0x2000000
set shmsys:shminfo_shmmin=1
set shmsys:shminfo_shmmni=256
set shmsys:shminfo_shmseg=256
set semsys:seminfo_semmap=256
set semsys:seminfo_semmni=512
set semsys:seminfo_semmns=512
set semsys:seminfo_semmsl=32
</programlisting>
You need to reboot for the changes to take effect. See also
<ulink url="http://sunsite.uakom.sk/sunworldonline/swol-09-1997/swol-09-insidesolaris.html"></ulink>
for information on shared memory under older versions of Solaris.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><systemitem class="osname">Solaris</> 2.10 (Solaris
10) and later</term>
<term><systemitem class="osname">OpenSolaris</></term>
<listitem>
<para>
In Solaris 10 and later, and OpenSolaris, the default shared memory and
semaphore settings are good enough for most
<productname>PostgreSQL</> applications. Solaris now defaults
to a <varname>SHMMAX</> of one-quarter of system <acronym>RAM</>.
To further adjust this setting, use a project setting associated
with the <literal>postgres</> user. For example, run the
following as <literal>root</>:
<programlisting>
projadd -c "PostgreSQL DB User" -K "project.max-shm-memory=(privileged,8GB,deny)" -U postgres -G postgres user.postgres
</programlisting>
</para>
<para>
This command adds the <literal>user.postgres</> project and
sets the shared memory maximum for the <literal>postgres</>
user to 8GB, and takes effect the next time that user logs
in, or when you restart <productname>PostgreSQL</> (not reload).
The above assumes that <productname>PostgreSQL</> is run by
the <literal>postgres</> user in the <literal>postgres</>
group. No server reboot is required.
</para>
<para>
Other recommended kernel setting changes for database servers which will
have a large number of connections are:
<programlisting>
project.max-shm-ids=(priv,32768,deny)
project.max-sem-ids=(priv,4096,deny)
project.max-msg-ids=(priv,4096,deny)
</programlisting>
</para>
<para>
Additionally, if you are running <productname>PostgreSQL</>
inside a zone, you may need to raise the zone resource usage
limits as well. See "Chapter2: Projects and Tasks" in the
<citetitle>System Administrator's Guide</> for more
information on <literal>projects</> and <command>prctl</>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><systemitem class="osname">UnixWare</>
<indexterm><primary>UnixWare</><secondary>IPC configuration</></>
</term>
<listitem>
<para>
On <productname>UnixWare</> 7, the maximum size for shared
memory segments is 512 kB in the default configuration.
To display the current value of <varname>SHMMAX</>, run:
<programlisting>
/etc/conf/bin/idtune -g SHMMAX
</programlisting>
which displays the current, default, minimum, and maximum
values. To set a new value for <varname>SHMMAX</>,
run:
<programlisting>
/etc/conf/bin/idtune SHMMAX <replaceable>value</>
</programlisting>
where <replaceable>value</> is the new value you want to use
(in bytes). After setting <varname>SHMMAX</>, rebuild the
kernel:
<programlisting>
/etc/conf/bin/idbuild -B
</programlisting>
and reboot.
</para>
</listitem>
</varlistentry>
</variablelist>
</sect2>
<sect2>
<title>Resource Limits</title>
<para>
Unix-like operating systems enforce various kinds of resource limits
that might interfere with the operation of your
<productname>PostgreSQL</productname> server. Of particular
importance are limits on the number of processes per user, the
number of open files per process, and the amount of memory available
to each process. Each of these have a <quote>hard</quote> and a
<quote>soft</quote> limit. The soft limit is what actually counts
but it can be changed by the user up to the hard limit. The hard
limit can only be changed by the root user. The system call
<function>setrlimit</function> is responsible for setting these
parameters. The shell's built-in command <command>ulimit</command>
(Bourne shells) or <command>limit</command> (<application>csh</>) is
used to control the resource limits from the command line. On
BSD-derived systems the file <filename>/etc/login.conf</filename>
controls the various resource limits set during login. See the
operating system documentation for details. The relevant
parameters are <varname>maxproc</varname>,
<varname>openfiles</varname>, and <varname>datasize</varname>. For
example:
<programlisting>
default:\
...
:datasize-cur=256M:\
:maxproc-cur=256:\
:openfiles-cur=256:\
...
</programlisting>
(<literal>-cur</literal> is the soft limit. Append
<literal>-max</literal> to set the hard limit.)
</para>
<para>
Kernels can also have system-wide limits on some resources.
<itemizedlist>
<listitem>
<para>
On <productname>Linux</productname>
<filename>/proc/sys/fs/file-max</filename> determines the
maximum number of open files that the kernel will support. It can
be changed by writing a different number into the file or by
adding an assignment in <filename>/etc/sysctl.conf</filename>.
The maximum limit of files per process is fixed at the time the
kernel is compiled; see
<filename>/usr/src/linux/Documentation/proc.txt</filename> for
more information.
</para>
</listitem>
</itemizedlist>
</para>
<para>
The <productname>PostgreSQL</productname> server uses one process
per connection so you should provide for at least as many processes
as allowed connections, in addition to what you need for the rest
of your system. This is usually not a problem but if you run
several servers on one machine things might get tight.
</para>
<para>
The factory default limit on open files is often set to
<quote>socially friendly</quote> values that allow many users to
coexist on a machine without using an inappropriate fraction of
the system resources. If you run many servers on a machine this
is perhaps what you want, but on dedicated servers you might want to
raise this limit.
</para>
<para>
On the other side of the coin, some systems allow individual
processes to open large numbers of files; if more than a few
processes do so then the system-wide limit can easily be exceeded.
If you find this happening, and you do not want to alter the
system-wide limit, you can set <productname>PostgreSQL</>'s <xref
linkend="guc-max-files-per-process"> configuration parameter to
limit the consumption of open files.
</para>
</sect2>
<sect2 id="linux-memory-overcommit">
<title>Linux Memory Overcommit</title>
<para>
In Linux 2.4 and later, the default virtual memory behavior is not
optimal for <productname>PostgreSQL</productname>. Because of the
way that the kernel implements memory overcommit, the kernel might
terminate the <productname>PostgreSQL</productname> postmaster (the
master server process) if the memory demands of either
<productname>PostgreSQL</productname> or another process cause the
system to run out of virtual memory.
</para>
<para>
If this happens, you will see a kernel message that looks like
this (consult your system documentation and configuration on where
to look for such a message):
<programlisting>
Out of Memory: Killed process 12345 (postgres).
</programlisting>
This indicates that the <filename>postgres</filename> process
has been terminated due to memory pressure.
Although existing database connections will continue to function
normally, no new connections will be accepted. To recover,
<productname>PostgreSQL</productname> will need to be restarted.
</para>
<para>
One way to avoid this problem is to run
<productname>PostgreSQL</productname> on a machine where you can
be sure that other processes will not run the machine out of
memory. If memory is tight, increasing the swap space of the
operating system can help avoid the problem, because the
out-of-memory (OOM) killer is invoked only when physical memory and
swap space are exhausted.
</para>
<para>
If <productname>PostgreSQL</productname> itself is the cause of the
system running out of memory, you can avoid the problem by changing
your configuration. In some cases, it may help to lower memory-related
configuration parameters, particularly
<link linkend="guc-shared-buffers"><varname>shared_buffers</></link>
and <link linkend="guc-work-mem"><varname>work_mem</></link>. In
other cases, the problem may be caused by allowing too many connections
to the database server itself. In many cases, it may be better to reduce
<link linkend="guc-max-connections"><varname>max_connections</></link>
and instead make use of external connection-pooling software.
</para>
<para>
On Linux 2.6 and later, it is possible to modify the
kernel's behavior so that it will not <quote>overcommit</> memory.
Although this setting will not prevent the <ulink
url="http://lwn.net/Articles/104179/">OOM killer</> from being invoked
altogether, it will lower the chances significantly and will therefore
lead to more robust system behavior. This is done by selecting strict
overcommit mode via <command>sysctl</command>:
<programlisting>
sysctl -w vm.overcommit_memory=2
</programlisting>
or placing an equivalent entry in <filename>/etc/sysctl.conf</>.
You might also wish to modify the related setting
<varname>vm.overcommit_ratio</>. For details see the kernel documentation
file <filename>Documentation/vm/overcommit-accounting</>.
</para>
<para>
Another approach, which can be used with or without altering
<varname>vm.overcommit_memory</>, is to set the process-specific
<firstterm>OOM score adjustment</> value for the postmaster process to
<literal>-1000</>, thereby guaranteeing it will not be targeted by the OOM
killer. The simplest way to do this is to execute
<programlisting>
echo -1000 > /proc/self/oom_score_adj
</programlisting>
in the postmaster's startup script just before invoking the postmaster.
Note that this action must be done as root, or it will have no effect;
so a root-owned startup script is the easiest place to do it. If you
do this, you should also set these environment variables in the startup
script before invoking the postmaster:
<programlisting>
export PG_OOM_ADJUST_FILE=/proc/self/oom_score_adj
export PG_OOM_ADJUST_VALUE=0
</programlisting>
These settings will cause postmaster child processes to run with the
normal OOM score adjustment of zero, so that the OOM killer can still
target them at need. You could use some other value for
<envar>PG_OOM_ADJUST_VALUE</> if you want the child processes to run
with some other OOM score adjustment. (<envar>PG_OOM_ADJUST_VALUE</>
can also be omitted, in which case it defaults to zero.) If you do not
set <envar>PG_OOM_ADJUST_FILE</>, the child processes will run with the
same OOM score adjustment as the postmaster, which is unwise since the
whole point is to ensure that the postmaster has a preferential setting.
</para>
<para>
Older Linux kernels do not offer <filename>/proc/self/oom_score_adj</>,
but may have a previous version of the same functionality called
<filename>/proc/self/oom_adj</>. This works the same except the disable
value is <literal>-17</> not <literal>-1000</>.
</para>
<note>
<para>
Some vendors' Linux 2.4 kernels are reported to have early versions
of the 2.6 overcommit <command>sysctl</command> parameter. However, setting
<literal>vm.overcommit_memory</> to 2
on a 2.4 kernel that does not have the relevant code will make
things worse, not better. It is recommended that you inspect
the actual kernel source code (see the function
<function>vm_enough_memory</> in the file <filename>mm/mmap.c</>)
to verify what is supported in your kernel before you try this in a 2.4
installation. The presence of the <filename>overcommit-accounting</>
documentation file should <emphasis>not</> be taken as evidence that the
feature is there. If in any doubt, consult a kernel expert or your
kernel vendor.
</para>
</note>
</sect2>
<sect2 id="linux-huge-pages">
<title>Linux huge pages</title>
<para>
Using huge pages reduces overhead when using large contiguous chunks of
memory, like <productname>PostgreSQL</productname> does. To enable this
feature in <productname>PostgreSQL</productname> you need a kernel
with <varname>CONFIG_HUGETLBFS=y</varname> and
<varname>CONFIG_HUGETLB_PAGE=y</varname>. You also have to tune the system
setting <varname>vm.nr_hugepages</varname>. To estimate the number of
necessary huge pages start <productname>PostgreSQL</productname> without
huge pages enabled and check the <varname>VmPeak</varname> value from the
proc file system:
<programlisting>
$ <userinput>head -1 /path/to/data/directory/postmaster.pid</userinput>
4170
$ <userinput>grep ^VmPeak /proc/4170/status</userinput>
VmPeak: 6490428 kB
</programlisting>
<literal>6490428</literal> / <literal>2048</literal>
(<varname>PAGE_SIZE</varname> is <literal>2MB</literal> in this case) are
roughly <literal>3169.154</literal> huge pages, so you will need at
least <literal>3170</literal> huge pages:
<programlisting>
$ <userinput>sysctl -w vm.nr_hugepages=3170</userinput>
</programlisting>
Sometimes the kernel is not able to allocate the desired number of huge
pages, so it might be necessary to repeat that command or to reboot. Don't
forget to add an entry to <filename>/etc/sysctl.conf</filename> to persist
this setting through reboots.
</para>
<para>
The default behavior for huge pages in
<productname>PostgreSQL</productname> is to use them when possible and
to fallback to normal pages when failing. To enforce the use of huge
pages, you can set
<link linkend="guc-huge-pages"><varname>huge_pages</varname></link>
to <literal>on</literal>. Note that in this case
<productname>PostgreSQL</productname> will fail to start if not enough huge
pages are available.
</para>
<para>
For a detailed description of the <productname>Linux</productname> huge
pages feature have a look
at <ulink url="https://www.kernel.org/doc/Documentation/vm/hugetlbpage.txt">https://www.kernel.org/doc/Documentation/vm/hugetlbpage.txt</ulink>.
</para>
</sect2>
</sect1>
<sect1 id="server-shutdown">
<title>Shutting Down the Server</title>
<indexterm zone="server-shutdown">
<primary>shutdown</>
</indexterm>
<para>
There are several ways to shut down the database server. You control
the type of shutdown by sending different signals to the master
<command>postgres</command> process.
<variablelist>
<varlistentry>
<term><systemitem>SIGTERM</systemitem><indexterm><primary>SIGTERM</></></term>
<listitem>
<para>
This is the <firstterm>Smart Shutdown</firstterm> mode.
After receiving <systemitem>SIGTERM</systemitem>, the server
disallows new connections, but lets existing sessions end their
work normally. It shuts down only after all of the sessions terminate.
If the server is in online backup mode, it additionally waits
until online backup mode is no longer active. While backup mode is
active, new connections will still be allowed, but only to superusers
(this exception allows a superuser to connect to terminate
online backup mode). If the server is in recovery when a smart
shutdown is requested, recovery and streaming replication will be
stopped only after all regular sessions have terminated.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><systemitem>SIGINT</systemitem><indexterm><primary>SIGINT</></></term>
<listitem>
<para>
This is the <firstterm>Fast Shutdown</firstterm> mode.
The server disallows new connections and sends all existing
server processes <systemitem>SIGTERM</systemitem>, which will cause them
to abort their current transactions and exit promptly. It then
waits for all server processes to exit and finally shuts down.
If the server is in online backup mode, backup mode will be
terminated, rendering the backup useless.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><systemitem>SIGQUIT</systemitem><indexterm><primary>SIGQUIT</></></term>
<listitem>
<para>
This is the <firstterm>Immediate Shutdown</firstterm> mode.
The server will send <systemitem>SIGQUIT</systemitem> to all child
processes and wait for them to terminate. Those that don't terminate
within 5 seconds, will be sent <systemitem>SIGKILL</systemitem> by the
master <command>postgres</command> process, which will then terminate
without further waiting. This will lead to recovery (by
replaying the WAL log) upon next start-up. This is recommended
only in emergencies.
</para>
</listitem>
</varlistentry>
</variablelist>
</para>
<para>
The <xref linkend="app-pg-ctl"> program provides a convenient
interface for sending these signals to shut down the server.
Alternatively, you can send the signal directly using <command>kill</>
on non-Windows systems.
The <acronym>PID</> of the <command>postgres</command> process can be
found using the <command>ps</command> program, or from the file
<filename>postmaster.pid</filename> in the data directory. For
example, to do a fast shutdown:
<screen>
$ <userinput>kill -INT `head -1 /usr/local/pgsql/data/postmaster.pid`</userinput>
</screen>
</para>
<important>
<para>
It is best not to use <systemitem>SIGKILL</systemitem> to shut down
the server. Doing so will prevent the server from releasing
shared memory and semaphores, which might then have to be done
manually before a new server can be started. Furthermore,
<systemitem>SIGKILL</systemitem> kills the <command>postgres</command>
process without letting it relay the signal to its subprocesses,
so it will be necessary to kill the individual subprocesses by hand as
well.
</para>
</important>
<para>
To terminate an individual session while allowing other sessions to
continue, use <function>pg_terminate_backend()</> (see <xref
linkend="functions-admin-signal-table">) or send a
<systemitem>SIGTERM</> signal to the child process associated with
the session.
</para>
</sect1>
<sect1 id="upgrading">
<title>Upgrading a <productname>PostgreSQL</> Cluster</title>
<indexterm zone="upgrading">
<primary>upgrading</primary>
</indexterm>
<indexterm zone="upgrading">
<primary>version</primary>
<secondary>compatibility</secondary>
</indexterm>
<para>
This section discusses how to upgrade your database data from one
<productname>PostgreSQL</> release to a newer one.
</para>
<para>
<productname>PostgreSQL</> major versions are represented by the
first two digit groups of the version number, e.g., 8.4.
<productname>PostgreSQL</> minor versions are represented by the
third group of version digits, e.g., 8.4.2 is the second minor
release of 8.4. Minor releases never change the internal storage
format and are always compatible with earlier and later minor
releases of the same major version number, e.g., 8.4.2 is compatible
with 8.4, 8.4.1 and 8.4.6. To update between compatible versions,
you simply replace the executables while the server is down and
restart the server. The data directory remains unchanged —
minor upgrades are that simple.
</para>
<para>
For <emphasis>major</> releases of <productname>PostgreSQL</>, the
internal data storage format is subject to change, thus complicating
upgrades. The traditional method for moving data to a new major version
is to dump and reload the database. Other methods are available,
as discussed below.
</para>
<para>
New major versions also typically introduce some user-visible
incompatibilities, so application programming changes might be required.
All user-visible changes are listed in the release notes (<xref
linkend="release">); pay particular attention to the section
labeled "Migration". If you are upgrading across several major
versions, be sure to read the release notes for each intervening
version.
</para>
<para>
Cautious users will want to test their client applications on the new
version before switching over fully; therefore, it's often a good idea to
set up concurrent installations of old and new versions. When
testing a <productname>PostgreSQL</> major upgrade, consider the
following categories of possible changes:
</para>
<variablelist>
<varlistentry>
<term>Administration</term>
<listitem>
<para>
The capabilities available for administrators to monitor and control
the server often change and improve in each major release.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>SQL</term>
<listitem>
<para>
Typically this includes new SQL command capabilities and not changes
in behavior, unless specifically mentioned in the release notes.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Library API</term>
<listitem>
<para>
Typically libraries like <application>libpq</> only add new
functionality, again unless mentioned in the release notes.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>System Catalogs</term>
<listitem>
<para>
System catalog changes usually only affect database management tools.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Server C-language API</term>
<listitem>
<para>
This involves changes in the backend function API, which is written
in the C programming language. Such changes affect code that
references backend functions deep inside the server.
</para>
</listitem>
</varlistentry>
</variablelist>
<sect2 id="upgrade-methods-pgdump">
<title>Upgrading Data via <application>pg_dump</></title>
<para>
To dump data from one major version of <productname>PostgreSQL</> and
reload it in another, you must use <application>pg_dump</>; file system
level backup methods will not work. (There are checks in place that prevent
you from using a data directory with an incompatible version of
<productname>PostgreSQL</productname>, so no great harm can be done by
trying to start the wrong server version on a data directory.)
</para>
<para>
It is recommended that you use the <application>pg_dump</> and
<application>pg_dumpall</> programs from the newer version of
<productname>PostgreSQL</>, to take advantage of enhancements
that might have been made in these programs. Current releases of the
dump programs can read data from any server version back to 7.0.
</para>
<para>
These instructions assume that your existing installation is under the
<filename>/usr/local/pgsql</> directory, and that the data area is in
<filename>/usr/local/pgsql/data</>. Substitute your paths
appropriately.
</para>
<procedure>
<step>
<para>
If making a backup, make sure that your database is not being updated.
This does not affect the integrity of the backup, but the changed
data would of course not be included. If necessary, edit the
permissions in the file <filename>/usr/local/pgsql/data/pg_hba.conf</>
(or equivalent) to disallow access from everyone except you.
See <xref linkend="client-authentication"> for additional information on
access control.
</para>
<para>
<indexterm>
<primary>pg_dumpall</primary>
<secondary>use during upgrade</secondary>
</indexterm>
To back up your database installation, type:
<screen>
<userinput>pg_dumpall > <replaceable>outputfile</></userinput>
</screen>
If you need to preserve OIDs (such as when using them as
foreign keys), then use the <option>-o</option> option when running
<application>pg_dumpall</>.
</para>
<para>
To make the backup, you can use the <application>pg_dumpall</application>
command from the version you are currently running. For best
results, however, try to use the <application>pg_dumpall</application>
command from <productname>PostgreSQL</productname> &version;,
since this version contains bug fixes and improvements over older
versions. While this advice might seem idiosyncratic since you
haven't installed the new version yet, it is advisable to follow
it if you plan to install the new version in parallel with the
old version. In that case you can complete the installation
normally and transfer the data later. This will also decrease
the downtime.
</para>
</step>
<step>
<para>
Shut down the old server:
<screen>
<userinput>pg_ctl stop</>
</screen>
On systems that have <productname>PostgreSQL</> started at boot time,
there is probably a start-up file that will accomplish the same thing. For
example, on a <systemitem class="osname">Red Hat Linux</> system one
might find that this works:
<screen>
<userinput>/etc/rc.d/init.d/postgresql stop</userinput>
</screen>
See <xref linkend="runtime"> for details about starting and
stopping the server.
</para>
</step>
<step>
<para>
If restoring from backup, rename or delete the old installation
directory. It is a good idea to rename the directory, rather than
delete it, in case you have trouble and need to revert to it. Keep
in mind the directory might consume significant disk space. To rename
the directory, use a command like this:
<screen>
<userinput>mv /usr/local/pgsql /usr/local/pgsql.old</>
</screen>
(Be sure to move the directory as a single unit so relative paths
remain unchanged.)
</para>
</step>
<step>
<para>
Install the new version of <productname>PostgreSQL</productname> as
outlined in <![%standalone-include[the next section.]]>
<![%standalone-ignore[<xref linkend="install-procedure">.]]>
</para>
</step>
<step>
<para>
Create a new database cluster if needed. Remember that you must
execute these commands while logged in to the special database user
account (which you already have if you are upgrading).
<programlisting>
<userinput>/usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data</>
</programlisting>
</para>
</step>
<step>
<para>
Restore your previous <filename>pg_hba.conf</> and any
<filename>postgresql.conf</> modifications.
</para>
</step>
<step>
<para>
Start the database server, again using the special database user
account:
<programlisting>
<userinput>/usr/local/pgsql/bin/postgres -D /usr/local/pgsql/data</>
</programlisting>
</para>
</step>
<step>
<para>
Finally, restore your data from backup with:
<screen>
<userinput>/usr/local/pgsql/bin/psql -d postgres -f <replaceable>outputfile</></userinput>
</screen>
using the <emphasis>new</> <application>psql</>.
</para>
</step>
</procedure>
<para>
The least downtime can be achieved by installing the new server in
a different directory and running both the old and the new servers
in parallel, on different ports. Then you can use something like:
<programlisting>
pg_dumpall -p 5432 | psql -d postgres -p 5433
</programlisting>
to transfer your data.
</para>
</sect2>
<sect2 id="upgrading-methods-other">
<title>Non-Dump Upgrade Methods</title>
<para>
The <link linkend="pgupgrade">pg_upgrade</link> module allows an
installation to be migrated in-place from one major
<productname>PostgreSQL</> version to the next. Upgrades can be
performed in minutes.
</para>
<para>
It is also possible to use certain replication methods, such as
<productname>Slony</>, to create a standby server with the updated version of
<productname>PostgreSQL</>. This is possible because Slony supports
replication between different major versions of
<productname>PostgreSQL</>. The standby can be on the same computer or
a different computer. Once it has synced up with the master server
(running the older version of <productname>PostgreSQL</>), you can
switch masters and make the standby the master and shut down the older
database instance. Such a switch-over results in only several seconds
of downtime for an upgrade.
</para>
</sect2>
</sect1>
<sect1 id="preventing-server-spoofing">
<title>Preventing Server Spoofing</title>
<indexterm zone="preventing-server-spoofing">
<primary>server spoofing</primary>
</indexterm>
<para>
While the server is running, it is not possible for a malicious user
to take the place of the normal database server. However, when the
server is down, it is possible for a local user to spoof the normal
server by starting their own server. The spoof server could read
passwords and queries sent by clients, but could not return any data
because the <varname>PGDATA</> directory would still be secure because
of directory permissions. Spoofing is possible because any user can
start a database server; a client cannot identify an invalid server
unless it is specially configured.
</para>
<para>
The simplest way to prevent spoofing for <literal>local</>
connections is to use a Unix domain socket directory (<xref
linkend="guc-unix-socket-directories">) that has write permission only
for a trusted local user. This prevents a malicious user from creating
their own socket file in that directory. If you are concerned that
some applications might still reference <filename>/tmp</> for the
socket file and hence be vulnerable to spoofing, during operating system
startup create a symbolic link <filename>/tmp/.s.PGSQL.5432</> that points
to the relocated socket file. You also might need to modify your
<filename>/tmp</> cleanup script to prevent removal of the symbolic link.
</para>
<para>
To prevent spoofing on TCP connections, the best solution is to use
SSL certificates and make sure that clients check the server's certificate.
To do that, the server
must be configured to accept only <literal>hostssl</> connections (<xref
linkend="auth-pg-hba-conf">) and have SSL key and certificate files
(<xref linkend="ssl-tcp">). The TCP client must connect using
<literal>sslmode=verify-ca</> or
<literal>verify-full</> and have the appropriate root certificate
file installed (<xref linkend="libq-ssl-certificates">).
</para>
</sect1>
<sect1 id="encryption-options">
<title>Encryption Options</title>
<indexterm zone="encryption-options">
<primary>encryption</primary>
</indexterm>
<para>
<productname>PostgreSQL</productname> offers encryption at several
levels, and provides flexibility in protecting data from disclosure
due to database server theft, unscrupulous administrators, and
insecure networks. Encryption might also be required to secure
sensitive data such as medical records or financial transactions.
</para>
<variablelist>
<varlistentry>
<term>Password Storage Encryption</term>
<listitem>
<para>
By default, database user passwords are stored as MD5 hashes, so
the administrator cannot determine the actual password assigned
to the user. If MD5 encryption is used for client authentication,
the unencrypted password is never even temporarily present on the
server because the client MD5-encrypts it before being sent
across the network.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Encryption For Specific Columns</term>
<listitem>
<para>
The <xref linkend="pgcrypto"> module allows certain fields to be
stored encrypted.
This is useful if only some of the data is sensitive.
The client supplies the decryption key and the data is decrypted
on the server and then sent to the client.
</para>
<para>
The decrypted data and the decryption key are present on the
server for a brief time while it is being decrypted and
communicated between the client and server. This presents a brief
moment where the data and keys can be intercepted by someone with
complete access to the database server, such as the system
administrator.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Data Partition Encryption</term>
<listitem>
<para>
Storage encryption can be performed at the file system level or the
block level. Linux file system encryption options include eCryptfs
and EncFS, while FreeBSD uses PEFS. Block level or full disk
encryption options include dm-crypt + LUKS on Linux and GEOM
modules geli and gbde on FreeBSD. Many other operating systems
support this functionality, including Windows.
</para>
<para>
This mechanism prevents unencrypted data from being read from the
drives if the drives or the entire computer is stolen. This does
not protect against attacks while the file system is mounted,
because when mounted, the operating system provides an unencrypted
view of the data. However, to mount the file system, you need some
way for the encryption key to be passed to the operating system,
and sometimes the key is stored somewhere on the host that mounts
the disk.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Encrypting Passwords Across A Network</term>
<listitem>
<para>
The <literal>MD5</> authentication method double-encrypts the
password on the client before sending it to the server. It first
MD5-encrypts it based on the user name, and then encrypts it
based on a random salt sent by the server when the database
connection was made. It is this double-encrypted value that is
sent over the network to the server. Double-encryption not only
prevents the password from being discovered, it also prevents
another connection from using the same encrypted password to
connect to the database server at a later time.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Encrypting Data Across A Network</term>
<listitem>
<para>
SSL connections encrypt all data sent across the network: the
password, the queries, and the data returned. The
<filename>pg_hba.conf</> file allows administrators to specify
which hosts can use non-encrypted connections (<literal>host</>)
and which require SSL-encrypted connections
(<literal>hostssl</>). Also, clients can specify that they
connect to servers only via SSL. <application>Stunnel</> or
<application>SSH</> can also be used to encrypt transmissions.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>SSL Host Authentication</term>
<listitem>
<para>
It is possible for both the client and server to provide SSL
certificates to each other. It takes some extra configuration
on each side, but this provides stronger verification of identity
than the mere use of passwords. It prevents a computer from
pretending to be the server just long enough to read the password
sent by the client. It also helps prevent <quote>man in the middle</>
attacks where a computer between the client and server pretends to
be the server and reads and passes all data between the client and
server.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Client-Side Encryption</term>
<listitem>
<para>
If the system administrator for the server's machine cannot be trusted,
it is necessary
for the client to encrypt the data; this way, unencrypted data
never appears on the database server. Data is encrypted on the
client before being sent to the server, and database results have
to be decrypted on the client before being used.
</para>
</listitem>
</varlistentry>
</variablelist>
</sect1>
<sect1 id="ssl-tcp">
<title>Secure TCP/IP Connections with SSL</title>
<indexterm zone="ssl-tcp">
<primary>SSL</primary>
</indexterm>
<para>
<productname>PostgreSQL</> has native support for using
<acronym>SSL</> connections to encrypt client/server communications
for increased security. This requires that
<productname>OpenSSL</productname> is installed on both client and
server systems and that support in <productname>PostgreSQL</> is
enabled at build time (see <xref linkend="installation">).
</para>
<para>
With <acronym>SSL</> support compiled in, the
<productname>PostgreSQL</> server can be started with
<acronym>SSL</> enabled by setting the parameter
<xref linkend="guc-ssl"> to <literal>on</> in
<filename>postgresql.conf</>. The server will listen for both normal
and <acronym>SSL</> connections on the same TCP port, and will negotiate
with any connecting client on whether to use <acronym>SSL</>. By
default, this is at the client's option; see <xref
linkend="auth-pg-hba-conf"> about how to set up the server to require
use of <acronym>SSL</> for some or all connections.
</para>
<para>
<productname>PostgreSQL</productname> reads the system-wide
<productname>OpenSSL</productname> configuration file. By default, this
file is named <filename>openssl.cnf</filename> and is located in the
directory reported by <literal>openssl version -d</>.
This default can be overridden by setting environment variable
<envar>OPENSSL_CONF</envar> to the name of the desired configuration file.
</para>
<para>
<productname>OpenSSL</productname> supports a wide range of ciphers
and authentication algorithms, of varying strength. While a list of
ciphers can be specified in the <productname>OpenSSL</productname>
configuration file, you can specify ciphers specifically for use by
the database server by modifying <xref linkend="guc-ssl-ciphers"> in
<filename>postgresql.conf</>.
</para>
<note>
<para>
It is possible to have authentication without encryption overhead by
using <literal>NULL-SHA</> or <literal>NULL-MD5</> ciphers. However,
a man-in-the-middle could read and pass communications between client
and server. Also, encryption overhead is minimal compared to the
overhead of authentication. For these reasons NULL ciphers are not
recommended.
</para>
</note>
<para>
To start in <acronym>SSL</> mode, files containing the server certificate
and private key must exist. By default, these files are expected to be
named <filename>server.crt</> and <filename>server.key</>, respectively, in
the server's data directory, but other names and locations can be specified
using the configuration parameters <xref linkend="guc-ssl-cert-file">
and <xref linkend="guc-ssl-key-file">.
On Unix systems, the permissions on <filename>server.key</filename> must
disallow any access to world or group; achieve this by the command
<command>chmod 0600 server.key</command>.
If the private key is protected with a passphrase, the
server will prompt for the passphrase and will not start until it has
been entered.
</para>
<para>
In some cases, the server certificate might be signed by an
<quote>intermediate</> certificate authority, rather than one that is
directly trusted by clients. To use such a certificate, append the
certificate of the signing authority to the <filename>server.crt</> file,
then its parent authority's certificate, and so on up to a certificate
authority, <quote>root</> or <quote>intermediate</>, that is trusted by
clients, i.e. signed by a certificate in the clients'
<filename>root.crt</filename> files.
</para>
<sect2 id="ssl-client-certificates">
<title>Using Client Certificates</title>
<para>
To require the client to supply a trusted certificate, place
certificates of the certificate authorities (<acronym>CA</acronym>s)
you trust in the file <filename>root.crt</filename> in the data
directory, set the parameter <xref linkend="guc-ssl-ca-file"> in
<filename>postgresql.conf</filename> to <literal>root.crt</literal>,
and set the <literal>clientcert</literal> parameter
to 1 on the appropriate <literal>hostssl</> line(s) in
<filename>pg_hba.conf</>.
A certificate will then be requested from the client during
SSL connection startup. (See <xref linkend="libpq-ssl"> for a
description of how to set up certificates on the client.) The server will
verify that the client's certificate is signed by one of the trusted
certificate authorities. If intermediate <acronym>CA</>s appear in
<filename>root.crt</filename>, the file must also contain certificate
chains to their root <acronym>CA</>s. Certificate Revocation List
(CRL) entries
are also checked if the parameter <xref linkend="guc-ssl-crl-file"> is set.
<!-- If this URL changes replace it with a URL to www.archive.org. -->
(See <ulink
url="http://h71000.www7.hp.com/DOC/83final/BA554_90007/ch04s02.html"></>
for diagrams showing SSL certificate usage.)
</para>
<para>
The <literal>clientcert</literal> option in <filename>pg_hba.conf</> is
available for all authentication methods, but only for rows specified as
<literal>hostssl</>. When <literal>clientcert</literal> is not specified
or is set to 0, the server will still verify presented client
certificates against its CA list, if one is configured,
— but it will not insist that a client certificate be presented.
</para>
<para>
Note that the server's <filename>root.crt</filename> lists the top-level
CAs that are considered trusted for signing client certificates.
In principle it need
not list the CA that signed the server's certificate, though in most cases
that CA would also be trusted for client certificates.
</para>
<para>
If you are setting up client certificates, you may wish to use
the <literal>cert</> authentication method, so that the certificates
control user authentication as well as providing connection security.
See <xref linkend="auth-cert"> for details.
</para>
</sect2>
<sect2 id="ssl-server-files">
<title>SSL Server File Usage</title>
<para>
<xref linkend="ssl-file-usage"> summarizes the files that are
relevant to the SSL setup on the server. (The shown file names are default
or typical names. The locally configured names could be different.)
</para>
<table id="ssl-file-usage">
<title>SSL Server File Usage</title>
<tgroup cols="3">
<thead>
<row>
<entry>File</entry>
<entry>Contents</entry>
<entry>Effect</entry>
</row>
</thead>
<tbody>
<row>
<entry><xref linkend="guc-ssl-cert-file"> (<filename>$PGDATA/server.crt</>)</entry>
<entry>server certificate</entry>
<entry>sent to client to indicate server's identity</entry>
</row>
<row>
<entry><xref linkend="guc-ssl-key-file"> (<filename>$PGDATA/server.key</>)</entry>
<entry>server private key</entry>
<entry>proves server certificate was sent by the owner; does not indicate
certificate owner is trustworthy</entry>
</row>
<row>
<entry><xref linkend="guc-ssl-ca-file"> (<filename>$PGDATA/root.crt</>)</entry>
<entry>trusted certificate authorities</entry>
<entry>checks that client certificate is
signed by a trusted certificate authority</entry>
</row>
<row>
<entry><xref linkend="guc-ssl-crl-file"> (<filename>$PGDATA/root.crl</>)</entry>
<entry>certificates revoked by certificate authorities</entry>
<entry>client certificate must not be on this list</entry>
</row>
</tbody>
</tgroup>
</table>
<para>
The files <filename>server.key</>, <filename>server.crt</>,
<filename>root.crt</filename>, and <filename>root.crl</filename>
(or their configured alternative names)
are only examined during server start; so you must restart
the server for changes in them to take effect.
</para>
</sect2>
<sect2 id="ssl-certificate-creation">
<title>Creating a Self-signed Certificate</title>
<para>
To create a quick self-signed certificate for the server, use the
following <productname>OpenSSL</productname> command:
<programlisting>
openssl req -new -text -out server.req
</programlisting>
Fill out the information that <application>openssl</> asks for. Make sure
you enter the local host name as <quote>Common Name</>; the challenge
password can be left blank. The program will generate a key that is
passphrase protected; it will not accept a passphrase that is less
than four characters long. To remove the passphrase (as you must if
you want automatic start-up of the server), run the commands:
<programlisting>
openssl rsa -in privkey.pem -out server.key
rm privkey.pem
</programlisting>
Enter the old passphrase to unlock the existing key. Now do:
<programlisting>
openssl req -x509 -in server.req -text -key server.key -out server.crt
</programlisting>
to turn the certificate into a self-signed certificate and to copy
the key and certificate to where the server will look for them.
Finally do:
<programlisting>
chmod og-rwx server.key
</programlisting>
because the server will reject the file if its permissions are more
liberal than this.
For more details on how to create your server private key and
certificate, refer to the <productname>OpenSSL</> documentation.
</para>
<para>
A self-signed certificate can be used for testing, but a certificate
signed by a certificate authority (<acronym>CA</>) (either one of the
global <acronym>CAs</> or a local one) should be used in production
so that clients can verify the server's identity. If all the clients
are local to the organization, using a local <acronym>CA</> is
recommended.
</para>
</sect2>
</sect1>
<sect1 id="ssh-tunnels">
<title>Secure TCP/IP Connections with <application>SSH</application> Tunnels</title>
<indexterm zone="ssh-tunnels">
<primary>ssh</primary>
</indexterm>
<para>
It is possible to use <application>SSH</application> to encrypt the network
connection between clients and a
<productname>PostgreSQL</productname> server. Done properly, this
provides an adequately secure network connection, even for non-SSL-capable
clients.
</para>
<para>
First make sure that an <application>SSH</application> server is
running properly on the same machine as the
<productname>PostgreSQL</productname> server and that you can log in using
<command>ssh</command> as some user. Then you can establish a secure
tunnel with a command like this from the client machine:
<programlisting>
ssh -L 63333:localhost:5432 [email protected]
</programlisting>
The first number in the <option>-L</option> argument, 63333, is the
port number of your end of the tunnel; it can be any unused port.
(IANA reserves ports 49152 through 65535 for private use.) The
second number, 5432, is the remote end of the tunnel: the port
number your server is using. The name or IP address between the
port numbers is the host with the database server you are going to
connect to, as seen from the host you are logging in to, which
is <literal>foo.com</literal> in this example. In order to connect
to the database server using this tunnel, you connect to port 63333
on the local machine:
<programlisting>
psql -h localhost -p 63333 postgres
</programlisting>
To the database server it will then look as though you are really
user <literal>joe</literal> on host <literal>foo.com</literal>
connecting to <literal>localhost</literal> in that context, and it
will use whatever authentication procedure was configured for
connections from this user and host. Note that the server will not
think the connection is SSL-encrypted, since in fact it is not
encrypted between the
<application>SSH</application> server and the
<productname>PostgreSQL</productname> server. This should not pose any
extra security risk as long as they are on the same machine.
</para>
<para>
In order for the
tunnel setup to succeed you must be allowed to connect via
<command>ssh</command> as <literal>[email protected]</literal>, just
as if you had attempted to use <command>ssh</command> to create a
terminal session.
</para>
<para>
You could also have set up the port forwarding as
<programlisting>
ssh -L 63333:foo.com:5432 [email protected]
</programlisting>
but then the database server will see the connection as coming in
on its <literal>foo.com</literal> interface, which is not opened by
the default setting <literal>listen_addresses =
'localhost'</literal>. This is usually not what you want.
</para>
<para>
If you have to <quote>hop</quote> to the database server via some
login host, one possible setup could look like this:
<programlisting>
ssh -L 63333:db.foo.com:5432 [email protected]
</programlisting>
Note that this way the connection
from <literal>shell.foo.com</literal>
to <literal>db.foo.com</literal> will not be encrypted by the SSH
tunnel.
SSH offers quite a few configuration possibilities when the network
is restricted in various ways. Please refer to the SSH
documentation for details.
</para>
<tip>
<para>
Several other applications exist that can provide secure tunnels using
a procedure similar in concept to the one just described.
</para>
</tip>
</sect1>
<sect1 id="event-log-registration">
<title>Registering <application>Event Log</> on <systemitem
class="osname">Windows</></title>
<indexterm zone="event-log-registration">
<primary>event log</primary>
<secondary>event log</secondary>
</indexterm>
<para>
To register a <systemitem class="osname">Windows</>
<application>event log</> library with the operating system,
issue this command:
<screen>
<userinput>regsvr32 <replaceable>pgsql_library_directory</>/pgevent.dll</>
</screen>
This creates registry entries used by the event viewer, under the default
event source named <literal>PostgreSQL</literal>.
</para>
<para>
To specify a different event source name (see
<xref linkend="guc-event-source">), use the <literal>/n</literal>
and <literal>/i</literal> options:
<screen>
<userinput>regsvr32 /n /i:<replaceable>event_source_name</> <replaceable>pgsql_library_directory</>/pgevent.dll</>
</screen>
</para>
<para>
To unregister the <application>event log</> library from
the operating system, issue this command:
<screen>
<userinput>regsvr32 /u [/i:<replaceable>event_source_name</>] <replaceable>pgsql_library_directory</>/pgevent.dll</>
</screen>
</para>
<note>
<para>
To enable event logging in the database server, modify
<xref linkend="guc-log-destination"> to include
<literal>eventlog</literal> in <filename>postgresql.conf</filename>.
</para>
</note>
</sect1>
</chapter>
|