SECOND AMENDMENT TO ASSIGNMENT OF LEASES AND RENTS

Exhibit 10.4

SECOND AMENDMENT TO ASSIGNMENT OF LEASES AND RENTS

Cross-Reference to:

Assignment of Leases and Rents

recorded in Liber     , Page     , and

First Amendment to Assignment of Leases

and Rents recorded in Liber     ,

Page     , Clerk/Register of Deeds,

Oakland County, MI

DC-19675 W. TEN MILE, LLC

a Delaware limited liability company,

as Grantor

AND

KEYBANK NATIONAL ASSOCIATION,

a national banking association, as Agent for Lenders

as Agent

DATED: AS OF July 19, 2012

County: Oakland

State: Michigan

Tax ID Numbers: 24-26-101-003 as to Parcel IB and 24-26-101-004 as to Parcel IC

SECOND AMENDMENT TO ASSIGNMENT OF LEASES AND RENTS

THIS SECOND AMENDMENT TO ASSIGNMENT OF LEASES AND RENTS (this “Amendment”) is made as of the 19th day of July, 2012, by and between DC-19675 W. TEN MILE, LLC, a Delaware limited liability company (“Grantor”), as assignor, having a mailing address of 4211 W. Boy Scout Boulevard, #500, Tampa, Florida 33607, and KEYBANK NATIONAL ASSOCIATION, as assignee, a national banking association (“KeyBank”), having a mailing address of 4900 Tiedeman Road, Brooklyn, Ohio 44144, Attn: Real Estate Capital Services, with a copy to KeyBank National Association, 1200 Abernathy Road, N.E., Suite 1550, Atlanta, Georgia 30328, Attn: Daniel Stegemoeller, as Agent (KeyBank, in its capacity as Agent, is hereinafter referred to as “Agent”) for itself and each other lender (collectively, the “Lenders”) which is or may hereafter become a party to the “Credit Agreement” (as hereinafter defined).

RECITALS

WHEREAS, Carter/Validus Operating Partnership, LP, a Delaware limited partnership (“Borrower”), KeyBank, as Agent and the Lenders from time to time a party thereto entered into that certain Credit Agreement dated as of March 30, 2012, as amended by that certain First Amendment to Credit Agreement dated as of May 8, 2012, and as further amended by that certain Second Amendment to Credit Agreement dated as of June 29, 2012 (as the same may be further varied, amended, restated, renewed, consolidated, extended or otherwise supplemented from time to time, the “Credit Agreement”);

WHEREAS, in connection with the Credit Agreement, Grantor entered into that certain Mortgage dated as of May 8, 2012, and recorded in Liber     , Page      with the Clerk/Register of Deeds, Oakland County, Michigan (the “Original Mortgage”) encumbering certain real property located in Oakland County, Michigan more particularly described on Exhibit “A” attached hereto and made a part hereof and certain personal property related thereto (said real and personal property, the “Property”);

WHEREAS, in connection with the Credit Agreement, Grantor entered into that certain Assignment of Leases and Rents dated as of May 8, 2012, and recorded in Liber     , Page      with the Clerk/Register of Deeds, Oakland County, Michigan (the “Original Assignment of Leases”); and

WHEREAS, Grantor and Agent entered into that certain First Amendment to Mortgage, dated as of June 29, 2012, and recorded in Liber     , Page      with the Clerk/Register of Deeds, Oakland County, Michigan (the “First Mortgage Amendment”) amending the Original Mortgage (the Original Mortgage, as amended by the First Mortgage Amendment, the “Mortgage”) and that certain First Amendment to Assignment of Leases and Rents, dated as of June 29, 2012, and recorded in Liber     , Page      with the Clerk/Register of Deeds, Oakland County, Michigan (the “First Assignment Amendment”) amending the Original Assignment of Leases (the Original Assignment of Leases, as amended by the First Assignment Amendment, the “Assignment of Leases”); and

WHEREAS, in connection with an increase of the Total Commitment from $40,000,000.00 to $55,000,000.00 under the Credit Agreement, the parties hereto desire to amend the Assignment of Leases to reflect said increase.

NOW, THEREFORE, for and in consideration of the sum of TEN and NO/100 DOLLARS ($10.00), and other good and valuable consideration, the receipt and sufficiency of which is hereby acknowledged, the parties hereto do hereby covenant and agree as follows:

1. Definitions. Capitalized terms used herein that are not otherwise defined herein shall have the meanings set forth in the Credit Agreement.

2. Modification of the Assignment of Leases. Grantor and Agent do hereby modify and amend the Assignment of Leases by deleting in its entirety subparagraph (a) appearing on page 2 of the Assignment of Leases, and inserting in lieu thereof the following subparagraph (a):

“(a) The debt evidenced by (i) those certain Revolving Credit Notes made by Borrower in the aggregate principal amount of Fifty-Five Million and No/100 Dollars ($55,000,000.00) to the order of Lenders, and (ii) that certain Swing Loan Note made by Borrower in the original principal face amount of Ten Million and No/100 Dollars ($10,000,000.00) to the order of KeyBank, each of which has been issued pursuant to the Credit Agreement and each of which is due and payable in full on or before March 30, 2015, unless extended as provided in the Credit Agreement, and (iii) each other note as may be issued under the Credit Agreement, each as originally executed, or if varied, extended, supplemented, consolidated, amended, replaced, renewed, modified or restated from time to time, as so varied, extended, supplemented, consolidated, amended, replaced, renewed, modified or restated (collectively, the “Note”);”

3. References to Assignment of Leases. All references in the Loan Documents to the Assignment of Leases shall be deemed a reference to the Assignment of Leases, as modified and amended herein.

4. Ratification. Except as hereinabove set forth, all terms, covenants, and provisions of the Assignment of Leases remain unaltered and in full force and effect, and the parties hereto do hereby expressly ratify and confirm the Assignment of Leases as modified and amended herein. Nothing in this Amendment or in any other document executed in connection herewith shall be deemed or construed to constitute, and there has not otherwise occurred, a novation, cancellation, satisfaction, release, extinguishment, or substitution of the indebtedness evidenced by the Notes or the other obligations of Grantor under the Loan Documents.

5. Amendment as Loan Document. This Amendment shall constitute a Loan Document.

6. Counterparts. This Amendment may be executed in any number of counterparts which shall together constitute but one and the same agreement.

7. Entire Agreement. This Amendment and the other Loan Documents (a) integrate all the terms and conditions set forth in or incidental to the Loan Documents, (b) supersede all

3

oral negotiations and prior writings with respect to the subject matter hereof, and (c) are intended by the parties as the final expression of the agreement with respect to the terms and conditions set forth in the Loan Documents and as the complete and exclusive statement of the terms agreed to by the parties. In the event of any conflict between the terms, conditions and provision of this Amendment and any other Loan Documents, the terms, conditions and provisions of this Amendment shall prevail.

8. Miscellaneous. This Amendment shall be construed and enforced in accordance with the laws of the state where the Property is located. Grantor shall pay all reasonable fees and expenses of Agent’s counsel in connection with the execution and delivery of this Amendment. This Amendment shall be binding upon and shall inure to the benefit of the parties hereto and their respective permitted successors, successors-in-title and assigns as provided in the Credit Agreement.

[Remainder of Page Intentionally Left Blank]

4

IN WITNESS WHEREOF, the parties hereto have hereto set their hands and affixed their seals as of the day and year first above written.

STATE OF FLORIDA

COUNTY OF HILLSBOROUGH

I, the undersigned, a Notary Public in and for said County, in said state, hereby certify that John E. Carter whose name as CEO of Carter Validus Mission Critical REIT, Inc., a Maryland corporation, which is the general partner of Carter/Validus Operating Partnership, LP, a Delaware limited partnership, which is the sole member of DC-19675 W. TEN MILE, LLC, a Delaware limited liability company, is signed to the foregoing instrument, and who is known to me, acknowledged before me on this day that, being informed of the contents of the instrument, he/she, as such officer and with full authority, executed the same voluntarily for and as the act of said officer on behalf of said company and stated that the representations therein contained are true.

GIVEN under my hand and Official Seal this 19 day of July, 2012.

[Signatures Continue on Next Page]

[Second Amendment to Assignment of Leases

Oakland County, MI]

I, the undersigned, a Notary Public in and for said county in the state aforesaid, do hereby certify that J.D. Gilbreath, the SR Vice President of KeyBank National Association, a national banking association, who is personally known to me to be the same person whose name is subscribed to the foregoing instrument, appeared before me this day in person and acknowledged that he signed and delivered the said instrument as such officer as his own free and voluntary act and as the free and voluntary act of said national banking association, for the uses and purposes set forth therein.

GIVEN under my hand and notarial seal, this 19 day of July, 2012.

My Commission Expires: June 29, 2012

PREPARED BY AND WHEN RECORDED, RETURN TO:

Brian T. Holmes, Esq.

McKenna Long & Aldridge LLP

303 Peachtree Street, N.E., Suite 5300

Atlanta, Georgia 30308

[Second Amendment to Assignment of Leases

Oakland County, MI]

EXHIBIT “A”

Real property in the City of Southfield, County of Oakland, State of Michigan, described as follows:

Parcel IB:

Part of the Northwest 1/4 of the Northwest 1/4 of Section 26, Town 1 North, Range 10 East, City of Southfield, Oakland County, Michigan, described as: Beginning at a point on the South line of Ten Mile Road (120 feet wide). Said point being North 89 degrees 59 minutes 45 seconds East 416.0 feet and South 0 degrees 00 minutes 15 seconds East 60.0 feet from the Northwest corner of Section 26, Town 1 North, Range 10 East; thence South 0 degrees 00 minutes 15 seconds East 118.00 feet; thence North 89 degrees 59 minutes 45 seconds East 125.0 feet; thence South 0 degrees 00 minutes 15 seconds East 94.75 feet; thence South 45 degrees 49 minutes 35 seconds East 252.68 feet; thence North 44 degrees 10 minutes 25 seconds East 72.0 feet; thence South 45 degrees 49 minutes 35 seconds East 50.15 feet; thence 177.95 feet along the Arc of a curve concave to the West, said curve having a Radius of 807.68 feet, a Central Angle of 12 degrees 37 minutes 25 seconds and whose Chord bears North 14 degrees 12 minutes 57 seconds East 177.59 feet; thence North 0 degrees 00 minutes 15 seconds West 200.0 feet; thence along the South line of Ten Mile Road, South 89 degrees 59 minutes 45 seconds West 435.98 feet to the point of beginning. Together with the right to use in common with others all easements for parking, ingress and egress, installation and maintenance of utilities and drainage facilities and use of all other common facilities, all as set forth in that certain instrument entitled “Declaration of Easements” dated June 25, 1968 and recorded in Liber 5225, Page 777, Oakland County Records as amended by instrument dated December 7, 1968 and recorded December 25, 1968 in Liber 5297, Page 465, Oakland County Records, and as further amended by instrument dated April 27, 2012 and recorded May 21, 2012 in Liber 44206, Page 87, Oakland County Records (as amended from time to time, the “Declaration”).

And together with a Generator Easement as granted in the Declaration over the property more particularly described as follows: Part of the Northwest 1/4 of Section 26, Township 1 North, Range 10 East, City of Southfield, Oakland County, Michigan, being more particularly described as follows; Commencing at the Northwest corner of Section 26; thence S89 degrees 57 minutes 31 seconds E, 415.61 feet (previously recorded as N89 degrees 59 minutes 45 seconds E, 415.00 feet) along the North line of Section 26; thence S00 degrees 00 minutes 15 seconds E, 178.24 feet (previously recorded at 178.00 feet); thence N89 degrees 59 minutes 45 seconds E, 125.00 feet; thence S00 degrees 00 minutes 15 seconds E, 94.75 feet; thence S45 degrees 49 minutes 35 seconds E, 252.68 feet; thence N44 degrees 10 minutes 25 seconds E, 42.00 feet to the point of beginning of the following described easement; thence N44 degrees 10 minutes 25 seconds E, 30.00 feet; thence S45 degrees 49 minutes 35 seconds E, 82.00 feet; thence S47 degrees 13 minutes 47 seconds W, 48.00 feet; thence N45 degrees 49 minutes 35 seconds W, 11.24 feet; thence N05 degrees 48 minutes 12 seconds E, 22.87 feet; thence N45 degrees 49 minutes 35 seconds W, 54.00 feet to the point of beginning.

A-1

Parcel IC:

Part of the Northwest 1/4 of the Northwest 1/4 of Section 26, Town 1 North, Range 10 East, City of Southfield, Oakland County, Michigan, described as: Beginning at a point on the South line of Ten Mile Road which is South 89 degrees 59 minutes 45 seconds West measured on the North line of Section 26, 1613.28 feet and South 00 degrees 00 minutes 15 seconds East 60.00 feet from the North 1/4 corner of said Section 26; thence South 89 degrees 59 minutes 45 seconds West along the South line of Ten Mile Road, 100.00 feet to a point; thence South 00 degrees 00 minutes 15 seconds East 200 feet to a point; thence North 89 degrees 59 minutes 45 seconds East, 100.00 feet to a point; thence North 00 degrees 00 minutes 15 seconds West, 200.0 feet to the place of beginning, except the Easterly 60 feet thereof.

Tax Parcel Identification Numbers: 24-26-101-003 as to Parcel IB and 24-26-101-004 as to Parcel IC

A-2