diff options
author | dan miller | 2007-10-20 02:49:29 +0000 |
---|---|---|
committer | dan miller | 2007-10-20 02:49:29 +0000 |
commit | e36d23a85ebff914d74bb541558c2b6082b78edb (patch) | |
tree | 54b58fdf162e78af64055282a6035c8d2443389d /libraries/sqlite/unix/sqlite-3.5.1/www/limits.tcl | |
parent | * Fixed an issue whereby avatar chat distances were being calculated against ... (diff) | |
download | opensim-SC_OLD-e36d23a85ebff914d74bb541558c2b6082b78edb.zip opensim-SC_OLD-e36d23a85ebff914d74bb541558c2b6082b78edb.tar.gz opensim-SC_OLD-e36d23a85ebff914d74bb541558c2b6082b78edb.tar.bz2 opensim-SC_OLD-e36d23a85ebff914d74bb541558c2b6082b78edb.tar.xz |
sqlite source (unix build) added to libraries
Diffstat (limited to 'libraries/sqlite/unix/sqlite-3.5.1/www/limits.tcl')
-rw-r--r-- | libraries/sqlite/unix/sqlite-3.5.1/www/limits.tcl | 318 |
1 files changed, 318 insertions, 0 deletions
diff --git a/libraries/sqlite/unix/sqlite-3.5.1/www/limits.tcl b/libraries/sqlite/unix/sqlite-3.5.1/www/limits.tcl new file mode 100644 index 0000000..cf85e38 --- /dev/null +++ b/libraries/sqlite/unix/sqlite-3.5.1/www/limits.tcl | |||
@@ -0,0 +1,318 @@ | |||
1 | # | ||
2 | # Run this script to generate the limits.html output file | ||
3 | # | ||
4 | set rcsid {$Id: limits.tcl,v 1.5 2007/08/09 00:00:26 drh Exp $} | ||
5 | source common.tcl | ||
6 | header {Implementation Limits For SQLite} | ||
7 | puts { | ||
8 | <h2>Limits In SQLite</h2> | ||
9 | |||
10 | <p> | ||
11 | "Limits" in the context of this article means sizes or | ||
12 | quantities that can not be exceeded. We are concerned | ||
13 | with things like the maximum number of bytes in a | ||
14 | BLOB or the maximum number of columns in a table. | ||
15 | </p> | ||
16 | |||
17 | <p> | ||
18 | SQLite was originally designed with a policy of avoiding | ||
19 | arbitrary limits. | ||
20 | Of course, every program that runs on a machine with finite | ||
21 | memory and disk space has limits of some kind. But in SQLite, | ||
22 | those limits | ||
23 | were not well defined. The policy was that if it would fit | ||
24 | in memory and you could count it with a 32-bit integer, then | ||
25 | it should work. | ||
26 | </p> | ||
27 | |||
28 | <p> | ||
29 | Unfortunately, the no-limits policy has been shown to create | ||
30 | problems. Because the upper bounds were not well | ||
31 | defined, they were not tested, and bugs (including possible | ||
32 | security exploits) were often found when pushing SQLite to | ||
33 | extremes. For this reason, newer versions of SQLite have | ||
34 | well-defined limits and those limits are tested as part of | ||
35 | the test suite. | ||
36 | </p> | ||
37 | |||
38 | <p> | ||
39 | This article defines what the limits of SQLite are and how they | ||
40 | can be customized for specific applications. The default settings | ||
41 | for limits are normally quite large and adequate for almost every | ||
42 | application. Some applications may what to increase a limit here | ||
43 | or there, but we expect such needs to be rare. More commonly, | ||
44 | an application might want to recompile SQLite with much lower | ||
45 | limits to avoid excess resource utilization in the event of | ||
46 | bug in higher-level SQL statement generators or to help thwart | ||
47 | attackers who inject malicious SQL statements. | ||
48 | </p> | ||
49 | } | ||
50 | proc limititem {title text} { | ||
51 | puts "<li><p><b>$title</b></p>\n$text</li>" | ||
52 | } | ||
53 | puts { | ||
54 | <ol> | ||
55 | } | ||
56 | |||
57 | limititem {Maximum length of a string or BLOB} { | ||
58 | <p> | ||
59 | The maximum number of bytes in a string or BLOB in SQLite is defined | ||
60 | by the preprocessor macro SQLITE_MAX_LENGTH. The default value | ||
61 | of this macro is 1 billion (1 thousand million or 1,000,000,000). | ||
62 | You can raise or lower this value at compile-time using a command-line | ||
63 | option like this: | ||
64 | </p> | ||
65 | |||
66 | <blockquote>-DSQLITE_MAX_LENGTH=123456789</blockquote> | ||
67 | |||
68 | <p> | ||
69 | The current implementation will only support a string or BLOB | ||
70 | length up to 2<small><sup>31</sup></small>-1 or 2147483647. And | ||
71 | some built-in functions such as hex() might fail well before that | ||
72 | point. In security-sensitive applications it is best not to | ||
73 | try to increase the maximum string and blob length. In fact, | ||
74 | you might do well to lower the maximum string and blob length | ||
75 | to something more in the range of a few million if that is | ||
76 | possible. | ||
77 | </p> | ||
78 | |||
79 | <p> | ||
80 | During part of SQLite's INSERT and SELECT processing, the complete | ||
81 | content of each row in the database is encoded as a single BLOB. | ||
82 | So the SQLITE_MAX_LENGTH parameter also determines the maximum | ||
83 | number of bytes in a row. | ||
84 | </p> | ||
85 | } | ||
86 | |||
87 | limititem {Maximum Number Of Columns} { | ||
88 | <p> | ||
89 | The SQLITE_MAX_COLUMN compile-time parameter is used to set an upper | ||
90 | bound on: | ||
91 | </p> | ||
92 | |||
93 | <ul> | ||
94 | <li>The number of columns in a table</li> | ||
95 | <li>The number of columns in an index</li> | ||
96 | <li>The number of columns in a view</li> | ||
97 | <li>The number of terms in the SET clause of an UPDATE statement</li> | ||
98 | <li>The number of columns in the result set of a SELECT statement</li> | ||
99 | <li>The number of terms in a GROUP BY or ORDER BY clause</li> | ||
100 | <li>The number of values in an INSERT statement</li> | ||
101 | </ul> | ||
102 | |||
103 | <p> | ||
104 | The default setting for SQLITE_MAX_COLUMN is 2000. You can change it | ||
105 | at compile time to values as large as 32676. You might be able to | ||
106 | redefine this value to be as large as billions, though nobody has ever | ||
107 | tried doing that so we do not know if it will work. On the other hand, there | ||
108 | are people who will argue that a well-normalized database design | ||
109 | will never need a value larger than about 100. | ||
110 | </p> | ||
111 | |||
112 | <p> | ||
113 | In most applications, the number of columns is small - a few dozen. | ||
114 | There are places in the SQLite code generator that use algorithms | ||
115 | that are O(N²) where N is the number of columns. | ||
116 | So if you redefine SQLITE_MAX_COLUMN to be a | ||
117 | really huge number and you generate SQL that uses a large number of | ||
118 | columns, you may find that | ||
119 | <a href="capi3ref.html#sqlite3_prepare_v2">sqlite3_prepare_v2()</a> | ||
120 | runs slowly. | ||
121 | } | ||
122 | |||
123 | limititem {Maximum Length Of An SQL Statement} { | ||
124 | <p> | ||
125 | The maximum number of bytes in the text of an SQL statement is | ||
126 | limited to SQLITE_MAX_SQL_LENGTH which defaults to 1000000. You | ||
127 | can redefine this limit to be as large as the smaller of SQLITE_MAX_LENGTH | ||
128 | and 1073741824. | ||
129 | </p> | ||
130 | |||
131 | <p> | ||
132 | If an SQL statement is limited to be a million bytes in length, then | ||
133 | obviously you will not be able to insert multi-million byte strings | ||
134 | by embedding them as literals inside of INSERT statements. But | ||
135 | you should not do that anyway. Use host parameters | ||
136 | for your data. Prepare short SQL statements like this: | ||
137 | </p> | ||
138 | |||
139 | <blockquote> | ||
140 | INSERT INTO tab1 VALUES(?,?,?); | ||
141 | </blockquote> | ||
142 | |||
143 | <p> | ||
144 | Then use the | ||
145 | <a href="capi3ref.html#sqlite3_bind_text">sqlite3_bind_XXXX()</a> functions | ||
146 | to bind your large string values to the SQL statement. The use of binding | ||
147 | obviates the need to escape quote characters in the string, reducing the | ||
148 | risk of SQL injection attacks. It is also runs faster since the large | ||
149 | string does not need to be parsed or copied as much. | ||
150 | </p> | ||
151 | } | ||
152 | |||
153 | limititem {Maximum Number Of Tables In A Join} { | ||
154 | <p> | ||
155 | SQLite does not support joins containing more than 64 tables. | ||
156 | This limit arises from the fact that the SQLite code generator | ||
157 | uses bitmaps with one bit per join-table in the query optimizer. | ||
158 | </p> | ||
159 | } | ||
160 | |||
161 | limititem {Maximum Depth Of An Expression Tree} { | ||
162 | <p> | ||
163 | SQLite parses expressions into a tree for processing. During | ||
164 | code generation, SQLite walks this tree recursively. The depth | ||
165 | of expression trees is therefore limited in order to avoid | ||
166 | using too much stack space. | ||
167 | </p> | ||
168 | |||
169 | <p> | ||
170 | The SQLITE_MAX_EXPR_DEPTH parameter determines the maximum expression | ||
171 | tree depth. If the value is 0, then no limit is enforced. The | ||
172 | current implementation has a default value of 1000. | ||
173 | </p> | ||
174 | } | ||
175 | |||
176 | limititem {Maximum Number Of Arguments On A Function} { | ||
177 | <p> | ||
178 | The SQLITE_MAX_FUNCTION_ARG parameter determines the maximum number | ||
179 | of parameters that can be passed to an SQL function. The default value | ||
180 | of this limit is 100. We know of no | ||
181 | technical reason why SQLite would not work with functions that have | ||
182 | millions of parameters. However, we suspect that anybody who tries | ||
183 | to invoke a function with millions of parameters is really | ||
184 | trying to find security exploits in systems that use SQLite, | ||
185 | not do useful work, | ||
186 | and so for that reason we have set this parameter relatively low. | ||
187 | } | ||
188 | |||
189 | limititem {Maximum Number Of Terms In A Compound SELECT Statement} { | ||
190 | <p> | ||
191 | A compound SELECT statement is two or more SELECT statements connected | ||
192 | by operators UNION, UNION ALL, EXCEPT, or INTERSECT. We call each | ||
193 | individual SELECT statement within a compound SELECT a "term". | ||
194 | </p> | ||
195 | |||
196 | <p> | ||
197 | The code generator in SQLite processes compound SELECT statements using | ||
198 | a recursive algorithm. In order to limit the size of the stack, we | ||
199 | therefore limit the number of terms in a compound SELECT. The maximum | ||
200 | number of terms is SQLITE_MAX_COMPOUND_SELECT which defaults to 500. | ||
201 | We think this is a generous allotment since in practice we almost | ||
202 | never see the number of terms in a compound select exceed single digits. | ||
203 | </p> | ||
204 | } | ||
205 | |||
206 | limititem {Maximum Length Of A LIKE Or GLOB Pattern} { | ||
207 | <p> | ||
208 | The pattern matching algorithm used in the default LIKE and GLOB | ||
209 | implementation of SQLite can exhibit O(N²) performance (where | ||
210 | N is the number of characters in the pattern) for certain pathological | ||
211 | cases. To avoid denial-of-service attacks from miscreants who are able | ||
212 | to specify their own LIKE or GLOB patterns, the length of the LIKE | ||
213 | or GLOB pattern is limited to SQLITE_MAX_LIKE_PATTERN_LENGTH bytes. | ||
214 | The default value of this limit is 50000. A modern workstation can | ||
215 | evaluate even a pathological LIKE or GLOB pattern of 50000 bytes | ||
216 | relatively quickly. The denial of service problem only comes into | ||
217 | play when the pattern length gets into millions of bytes. Nevertheless, | ||
218 | since most useful LIKE or GLOB patterns are at most a few dozen bytes | ||
219 | in length, paranoid application developers may want to reduce this | ||
220 | parameter to something in the range of a few hundred if they know that | ||
221 | external users are able to generate arbitrary patterns. | ||
222 | </p> | ||
223 | } | ||
224 | |||
225 | limititem {Maximum Number Of Host Parameters In A Single SQL Statement} { | ||
226 | <p> | ||
227 | A host parameter is a place-holder in an SQL statement that is filled | ||
228 | in using one of the | ||
229 | <a href="capi3ref.html#sqlite3_bind_blob">sqlite3_bind_XXXX()</a> interfaces. | ||
230 | Many SQL programmers are familiar with using a question mark ("?") as a | ||
231 | host parameter. SQLite also supports named host parameters prefaced | ||
232 | by ":", "$", or "@" and numbered host parameters of the form "?123". | ||
233 | </p> | ||
234 | |||
235 | <p> | ||
236 | Each host parameter in an SQLite statement is assigned a number. The | ||
237 | numbers normally begin with 1 and increase by one with each new | ||
238 | parameter. However, when the "?123" form is used, the host parameter | ||
239 | number is the number that follows the question mark. | ||
240 | </p> | ||
241 | |||
242 | <p> | ||
243 | The maximum value of a host parameter number is SQLITE_MAX_VARIABLE_NUMBER. | ||
244 | This setting defaults to 999. | ||
245 | </p> | ||
246 | } | ||
247 | |||
248 | limititem {Maximum Number Of Attached Databases} { | ||
249 | <p> | ||
250 | The <a href="lang_attach.html">ATTACH</a> statement is an SQLite extension | ||
251 | that allows two or more databases to be associated to the same database | ||
252 | connection and to operate as if they were a single database. The number | ||
253 | of simulataneously attached databases is limited to SQLITE_MAX_ATTACHED | ||
254 | which is set to 10 by default. | ||
255 | The code generator in SQLite uses bitmaps | ||
256 | to keep track of attached databases. That means that the number of | ||
257 | attached databases cannot be increased above 30 on a 32-bit machine | ||
258 | or 62 on a 64-bit machine. | ||
259 | } | ||
260 | |||
261 | limititem {Maximum Database Page Size} { | ||
262 | <p> | ||
263 | An SQLite database file is organized as pages. The size of each | ||
264 | page is a power of 2 between 512 and SQLITE_MAX_PAGE_SIZE. | ||
265 | The default value for SQLITE_MAX_PAGE_SIZE is 32768. The current | ||
266 | implementation will not support a larger value. | ||
267 | </p> | ||
268 | |||
269 | <p> | ||
270 | It used to be the case that SQLite would allocate some stack | ||
271 | structures whose size was proportional to the maximum page size. | ||
272 | For this reason, SQLite would sometimes be compiled with a smaller | ||
273 | maximum page size on embedded devices with limited stack memory. But | ||
274 | more recent versions of SQLite put these large structures on the | ||
275 | heap, not on the stack, so reducing the maximum page size is no | ||
276 | longer necessary on embedded devices. | ||
277 | </p> | ||
278 | } | ||
279 | |||
280 | limititem {Maximum Number Of Pages In A Database File} { | ||
281 | <p> | ||
282 | SQLite is able to limit the size of a database file to prevent | ||
283 | the database file from growing too large and consuming too much | ||
284 | disk or flash space. | ||
285 | The SQLITE_MAX_PAGE_COUNT parameter, which is normally set to | ||
286 | 1073741823, is the maximum number of pages allowed in a single | ||
287 | database file. An attempt to insert new data that would cause | ||
288 | the database file to grow larger than this will return | ||
289 | SQLITE_FULL. | ||
290 | </p> | ||
291 | |||
292 | <p> | ||
293 | The <a href="pragma.html#pragma_max_page_count"> | ||
294 | max_page_count PRAGMA</a> can be used to raise or lower this | ||
295 | limit at run-time. | ||
296 | </p> | ||
297 | |||
298 | <p> | ||
299 | Note that the transaction processing in SQLite requires two bits | ||
300 | of heap memory for every page in the database file. For databases | ||
301 | of a few megabytes in size, this amounts to only a few hundred | ||
302 | bytes of heap memory. But for gigabyte-sized databases the amount | ||
303 | of heap memory required is getting into the kilobyte range and | ||
304 | for terabyte-sized databases, megabytes of heap memory must be | ||
305 | allocated and zeroed at each transaction. SQLite will | ||
306 | support very large databases in theory, but the current implementation | ||
307 | is optimized for the common SQLite use cases of embedded devices | ||
308 | and persistent stores for desktop applications. In other words, | ||
309 | SQLite is designed for use with databases sized in kilobytes or | ||
310 | megabytes not gigabytes. If you are building an application to | ||
311 | work with databases that are hundreds of gigabytes or more | ||
312 | in size, then you should perhaps consider using a different database | ||
313 | engine that is explicitly designed for such large data sets. | ||
314 | </p> | ||
315 | } | ||
316 | |||
317 | puts {</ol>} | ||
318 | footer $rcsid | ||