Home All Groups Group Topic Archive Search About

Rolling back bcp takes forever. Shouldn't.



Author
29 Jun 2009 9:14 PM
Snake
I had to kill a SS2008 bcp-in because of space issues with the log file
device.  I believe the bcp had actually loaded the rows but had moved-on to
creating the Clustered index. The flat-file had 147,000,000 rows @35 gig. It
is my understanding that bcp loads the database pages and writes them to disk
very quickly and logs the space allocations in the db log file.  What I don't
understand is why it takes so long to roll-back (hours) when all it should be
doing is putting the allocated extents back into the free pool and leave the
data in-place as trash.  Obviously I am wrong or the rollback would not take
so long.  The log file expanded to 49 gig before I killed the transaction. 
Obviously more than space allocation was being logged.  Perhaps it was the
attempt to create the Clustered Index and because of that the rollback is
lengthy. 

I would like to hear from someone who really understands what is going on.

Thanks,

Michael

Author
29 Jun 2009 9:30 PM
Linchi Shea
Well, there is a difference between fast bcp and 'slow' bcp in that not all
bcp will result in minimally-logged operations. If you have data and indexes
in a table, most likely your bcp will end up being a row-by-row operation and
will be logged as such.

BOL has more info on the prerequisites for minimal logging in bulk import: ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/udb9/html/bd1dac6b-6ef8-4735-ad4e-67bb42dc4f66.htm

Linchi

Show quoteHide quote
"Snake" wrote:

> I had to kill a SS2008 bcp-in because of space issues with the log file
> device.  I believe the bcp had actually loaded the rows but had moved-on to
> creating the Clustered index. The flat-file had 147,000,000 rows @35 gig. It
> is my understanding that bcp loads the database pages and writes them to disk
> very quickly and logs the space allocations in the db log file.  What I don't
> understand is why it takes so long to roll-back (hours) when all it should be
> doing is putting the allocated extents back into the free pool and leave the
> data in-place as trash.  Obviously I am wrong or the rollback would not take
> so long.  The log file expanded to 49 gig before I killed the transaction. 
> Obviously more than space allocation was being logged.  Perhaps it was the
> attempt to create the Clustered Index and because of that the rollback is
> lengthy. 
>
> I would like to hear from someone who really understands what is going on.
>
> Thanks,
>
> Michael
Are all your drivers up to date? click for free checkup

Author
29 Jun 2009 9:33 PM
Linchi Shea
Also, check out this whitepaper:

http://msdn.microsoft.com/en-us/library/dd425070.aspx

It has more info on minimal logging.

Linchi

Show quoteHide quote
"Linchi Shea" wrote:

> Well, there is a difference between fast bcp and 'slow' bcp in that not all
> bcp will result in minimally-logged operations. If you have data and indexes
> in a table, most likely your bcp will end up being a row-by-row operation and
> will be logged as such.
>
> BOL has more info on the prerequisites for minimal logging in bulk import:
> ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/udb9/html/bd1dac6b-6ef8-4735-ad4e-67bb42dc4f66.htm
>
> Linchi
>
> "Snake" wrote:
>
> > I had to kill a SS2008 bcp-in because of space issues with the log file
> > device.  I believe the bcp had actually loaded the rows but had moved-on to
> > creating the Clustered index. The flat-file had 147,000,000 rows @35 gig. It
> > is my understanding that bcp loads the database pages and writes them to disk
> > very quickly and logs the space allocations in the db log file.  What I don't
> > understand is why it takes so long to roll-back (hours) when all it should be
> > doing is putting the allocated extents back into the free pool and leave the
> > data in-place as trash.  Obviously I am wrong or the rollback would not take
> > so long.  The log file expanded to 49 gig before I killed the transaction. 
> > Obviously more than space allocation was being logged.  Perhaps it was the
> > attempt to create the Clustered Index and because of that the rollback is
> > lengthy. 
> >
> > I would like to hear from someone who really understands what is going on.
> >
> > Thanks,
> >
> > Michael
Author
29 Jun 2009 9:47 PM
Snake
Thanks, the white paper covers things quite well. 

Show quoteHide quote
"Linchi Shea" wrote:

> Also, check out this whitepaper:
>
> http://msdn.microsoft.com/en-us/library/dd425070.aspx
>
> It has more info on minimal logging.
>
> Linchi
>
> "Linchi Shea" wrote:
>
> > Well, there is a difference between fast bcp and 'slow' bcp in that not all
> > bcp will result in minimally-logged operations. If you have data and indexes
> > in a table, most likely your bcp will end up being a row-by-row operation and
> > will be logged as such.
> >
> > BOL has more info on the prerequisites for minimal logging in bulk import:
> > ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/udb9/html/bd1dac6b-6ef8-4735-ad4e-67bb42dc4f66.htm
> >
> > Linchi
> >
> > "Snake" wrote:
> >
> > > I had to kill a SS2008 bcp-in because of space issues with the log file
> > > device.  I believe the bcp had actually loaded the rows but had moved-on to
> > > creating the Clustered index. The flat-file had 147,000,000 rows @35 gig. It
> > > is my understanding that bcp loads the database pages and writes them to disk
> > > very quickly and logs the space allocations in the db log file.  What I don't
> > > understand is why it takes so long to roll-back (hours) when all it should be
> > > doing is putting the allocated extents back into the free pool and leave the
> > > data in-place as trash.  Obviously I am wrong or the rollback would not take
> > > so long.  The log file expanded to 49 gig before I killed the transaction. 
> > > Obviously more than space allocation was being logged.  Perhaps it was the
> > > attempt to create the Clustered Index and because of that the rollback is
> > > lengthy. 
> > >
> > > I would like to hear from someone who really understands what is going on.
> > >
> > > Thanks,
> > >
> > > Michael
Author
29 Jun 2009 9:37 PM
Erland Sommarskog
Snake (Sn***@discussions.microsoft.com) writes:
> I had to kill a SS2008 bcp-in because of space issues with the log file
> device.  I believe the bcp had actually loaded the rows but had moved-on
> to creating the Clustered index. The flat-file had 147,000,000 rows @35
> gig. It is my understanding that bcp loads the database pages and writes
> them to disk very quickly and logs the space allocations in the db log
> file.  What I don't understand is why it takes so long to roll-back
> (hours) when all it should be doing is putting the allocated extents
> back into the free pool and leave the data in-place as trash.  Obviously
> I am wrong or the rollback would not take so long.  The log file
> expanded to 49 gig before I killed the transaction.  Obviously more than
> space allocation was being logged.  Perhaps it was the attempt to create
> the Clustered Index and because of that the rollback is lengthy. 

For BCP to use minimal recovery a couple of conditions must be fulfilled:

1) The database must be in simple or bulk-logged recovery.
2) You must use the TABLOCK hint.
3) If the table has a clustered index, it must be empty prior to the
   load.

It sounds that you did not fulfil of these, and that the operation was
fully logged.

The rollback will take a while. :-). Possibly it goes faster if you
restart SQL Server, in which case the rollback will be part of recovery.

--
Erland Sommarskog, SQL Server MVP, esq***@sommarskog.se

Links for SQL Server Books Online:
SQL 2008: http://msdn.microsoft.com/en-us/sqlserver/cc514207.aspx
SQL 2005: http://msdn.microsoft.com/en-us/sqlserver/bb895970.aspx
SQL 2000: http://www.microsoft.com/sql/prodinfo/previousversions/books.mspx

Bookmark and Share