AMENDMENT TO REGISTRATION RIGHTS AGREEMENT

Exhibit 4(e)

Execution Version

AMENDMENT TO REGISTRATION RIGHTS AGREEMENT

AMENDMENT TO REGISTRATION RIGHTS AGREEMENT, dated as of March 28, 2006 (this “Amendment”), to the Registration Rights Agreement, dated as of August 28, 2001 (as amended, supplemented or otherwise modified from time to time, the “Registration Rights Agreement”), by and among Imperial Sugar Company, a Texas corporation (the “Company”), and the other parties listed on the signature pages thereto (together with their successors and assigns, the “Holders”).

WITNESSETH:

WHEREAS, the parties hereto desire that certain provisions of the Registration Rights Agreement be amended as set forth below.

NOW THEREFORE, the parties hereto hereby agree as follows:

SECTION 1. Definitions. Terms defined in the Registration Rights Agreement and used herein shall have the meanings given to them in the Registration Rights Agreement.

SECTION 2. Amendments to the Registration Rights Agreement.

(a) Amendments to Section 1. Section 1 of the Registration Rights Agreement is hereby amended by adding the following new defined terms in proper alphabetical order:

“Demand Party” has the meaning set forth in Section 2.1(a) of this Agreement.

“Registration Request” has the meaning set forth in Section 2.1(a) of this Agreement.

(b) Amendments to Section 2.1(a). Section 2.1(a) of the Registration Rights Agreement is hereby amended by deleting the first two sentences thereof and substituting in lieu thereof the following:

“Upon the written request of Lehman Brothers, Inc. (the “Demand Party”) at any time on or before December 31, 2007 requesting that the Company effect the registration under the Securities Act of the Demand Party’s Registrable Shares (the “Registration Request”), the Company shall, as soon as reasonably practicable, use commercially reasonable efforts to file with the Commission, at the Company’s expense, a “shelf” registration statement on any appropriate form pursuant to Rule 415 under the Securities Act covering all Registrable Shares requested to be included in the registration by the Demand Party in the Registration Request (the “Shelf Registration Statement”); provided that, in no event shall the Company be required to effect more than one registration pursuant to this Section 2.1. The Company shall use commercially reasonable efforts to have the Shelf Registration Statement initially declared effective within 90 days after the

date of the Registration Request and to keep the Shelf Registration Statement continuously effective until the earliest of (i) such time as all shares of Registrable Shares have been sold thereunder, (ii) three years following the date on which the Shelf Registration Statement is declared effective, or (iii) such time as all of the Registrable Shares can be sold by the Electing Holders thereof under Rule 144 of the Securities Act or any successor or similar rule or provision without regard to volume limitations (the “Shelf Registration Period”).”

(c) Amendments to Section 2.1(b). Section 2.1(b) of the Registration Rights Agreement is hereby amended as follows:

(i) by deleting the first sentence therein and substituting in lieu thereof the following:

“Any Holder, including the Demand Party, desiring to include Registrable Shares in the Shelf Registration Statement must return a completed Notice and Questionnaire to the Company no later than 30 days after the date of the Registration Request.”

(ii) by deleting the reference to “the date of this Agreement” in the last sentence therein and substituting in lieu thereof “the date of the Registration Request”.

(d) Amendments to the Notice and Questionnaire. The form of Notice of Registration Statement and Selling Securityholder Questionnaire contained in Exhibit A to the Registration Rights Agreement is hereby amended by deleting the references therein to “the date of the Registration Rights Agreement” and substituting in lieu thereof “the date of the Registration Request”.

SECTION 3. Continued Force and Effect. Except as expressly amended hereby, the Registration Rights Agreement remains unchanged, and as amended hereby the Registration Rights Agreement remains in full force and effect.

SECTION 4. Governing Law. This Amendment shall be construed and enforced in accordance with and governed by the laws of the State of Texas, without giving effect to the conflicts of law principles thereof.

SECTION 5. Counterparts. This Amendment may be executed in any number of counterparts, each of which shall be deemed an original, but all such counterparts shall together constitute one and the same instrument.

IN WITNESS WHEREOF, the undersigned have executed this Amendment as of the date first above written: