@@ -238,7 +238,7 @@ processing will be sufficient.
238
238
Loading CSV Files into Oracle Database
239
239
======================================
240
240
241
- The :meth: `Cursor.executemany() ` method and `csv module
241
+ The :meth: `Cursor.executemany() ` method and Python's `csv module
242
242
<https://docs.python.org/3/library/csv.html#module-csv> `__ can be used to
243
243
efficiently load CSV (Comma Separated Values) files. For example, consider the
244
244
file ``data.csv ``::
@@ -255,21 +255,24 @@ And the schema:
255
255
256
256
create table test (id number, name varchar2(25));
257
257
258
- Instead of looping through each line of the CSV file and inserting it
259
- individually, you can insert batches of records using
260
- :meth: `Cursor.executemany() `:
258
+ Data loading can be done in batches of records since the number of records may
259
+ prevent all data being inserted at once:
261
260
262
261
.. code-block :: python
263
262
264
263
import cx_Oracle
265
264
import csv
266
265
267
- . . .
268
-
269
- # Predefine the memory areas to match the table definition
266
+ # Predefine the memory areas to match the table definition.
267
+ # This can improve performance by avoiding memory reallocations.
268
+ # Here, one parameter is passed for each of the columns.
269
+ # "None" is used for the ID column, since the size of NUMBER isn't
270
+ # variable. The "25" matches the maximum expected data size for the
271
+ # NAME column
270
272
cursor.setinputsizes(None , 25 )
271
273
272
- # Adjust the batch size to meet your memory and performance requirements
274
+ # Adjust the number of rows to be inserted in each iteration
275
+ # to meet your memory and performance requirements
273
276
batch_size = 10000
274
277
275
278
with open (' testsp.csv' , ' r' ) as csv_file:
@@ -284,3 +287,8 @@ individually, you can insert batches of records using
284
287
if data:
285
288
cursor.executemany(sql, data)
286
289
con.commit()
290
+
291
+
292
+ Depending on data sizes and business requirements, database changes such as
293
+ temporarily disabling redo logging on the table, or disabling indexes may also
294
+ be beneficial.
0 commit comments