ADMINISTRATION AGREEMENT

Exhibit 10.3

ADMINISTRATION AGREEMENT

AGREEMENT (this “Agreement”) made as of this 4th day of December, 2012, by and between WhiteHorse Finance, Inc., a Delaware corporation (the “Company”), and H.I.G. WhiteHorse Administration, LLC, a Delaware limited liability company (the “Administrator”).

W I T N E S S E T H:

WHEREAS, the Company is a closed-end non-diversified management investment company that has filed a notice with the Securities and Exchange Commission that has elected to be treated as a business development company under the Investment Company Act of 1940, as amended (the “Investment Company Act”);

WHEREAS, the Company desires to retain the Administrator to provide administrative services to the Company in the manner and on the terms hereinafter set forth; and

WHEREAS, the Administrator is willing to provide administrative services to the Company on the terms and conditions hereafter set forth.

NOW, THEREFORE, in consideration of the premises and the covenants hereinafter contained and for other good and valuable consideration, the receipt and adequacy of which is hereby acknowledged, the Company and the Administrator hereby agree as follows:

(a) Employment of Administrator. The Company hereby employs the Administrator to act as administrator of the Company, and to furnish, or arrange for others to furnish, the administrative services, personnel and facilities described below, subject to review by and the overall control of the Board of Directors of the Company, for the period and on the terms and conditions set forth in this Agreement. The Administrator hereby accepts such employment and agrees during such period to render, or arrange for the rendering of, such services and to assume the obligations herein set forth, subject to the reimbursement of costs and expenses provided for below. The Administrator and such others shall for all purposes herein be deemed to be independent contractors and shall, unless otherwise expressly provided or authorized herein, have no authority to act for or represent the Company in any way or otherwise be deemed agents of the Company.

(b) Services. The Administrator shall perform (or oversee, or arrange for, the performance of) the administrative services necessary for the operation of the Company. Without limiting the generality of the foregoing, the Administrator shall provide the Company with office facilities, equipment, clerical, bookkeeping and record keeping services at such facilities and such other services as the Administrator, subject to review by the Board of Directors of the Company, shall from time to time determine to be necessary or useful to perform its obligations under this Agreement. The Administrator shall also, on behalf of the Company, conduct relations with custodians, depositories, transfer agents, dividend disbursing agents, other stockholder servicing agents, accountants, attorneys, underwriters, brokers and dealers, corporate fiduciaries, insurers, banks and such other persons in any such other capacity deemed to be necessary or desirable. The Administrator shall make reports to the directors of the Company (the “Directors”)

of its performance of obligations hereunder and furnish advice and recommendations with respect to such other aspects of the business and affairs of the Company as it shall determine to be desirable; provided that nothing herein shall be construed to require the Administrator to, and the Administrator shall not, provide any advice or recommendation relating to securities and other assets that the Company should purchase, retain or sell or any other investment advisory services to the Company. The Administrator shall be responsible for the financial and other records that the Company is required to maintain and shall prepare reports to stockholders, and reports and other materials filed with the Securities and Exchange Commission (the “SEC”). The Administrator shall provide on the Company’s behalf significant managerial assistance to those portfolio companies to which the Company is required to provide such assistance. In addition, the Administrator shall assist the Company in determining and publishing the Company’s net asset value, oversee the preparation and filing of the Company’s tax returns, and the printing and dissemination of reports to stockholders of the Company and generally oversee the payment of the Company’s expenses and the performance of administrative and professional services rendered to the Company by others.

The Administrator agrees to maintain and keep all books, accounts and other records of the Company that relate to activities performed by the Administrator hereunder and, if required by the Investment Company Act, shall maintain and keep such books, accounts and records in accordance with such Act. In compliance with the requirements of Rule 31a-3 under the Investment Company Act, the Administrator agrees that all records that it maintains for the Company shall at all times remain the property of the Company, shall be readily accessible during normal business hours and shall be promptly surrendered upon the termination of the Agreement or otherwise on written request. The Administrator further agrees that all records that it maintains for the Company pursuant to Rule 31a-1 under the Investment Company Act shall be preserved for the periods prescribed by Rule 31a-2 under the Investment Company Act unless any such records are earlier surrendered as provided above. Records shall be surrendered in usable machine-readable form. The Administrator shall have the right to retain copies of such records, subject to observance of its confidentiality obligations under this Agreement.

Each of the parties hereto agrees that it shall treat confidentially the terms and conditions of this Agreement and all information provided by each party to the other regarding its business and operations. All confidential information provided by a party hereto, including nonpublic personal information pursuant to Regulation S-P of the SEC, shall be used by any other party hereto solely for the purpose of rendering services pursuant to this Agreement and, except as may be required in carrying out this Agreement, shall not be disclosed to any third party, without the prior consent of such providing party. The foregoing shall not apply to any information that is publicly available when provided or which thereafter becomes publicly available other than through a breach of this Agreement, or that is required to be disclosed by any regulatory authority, any authority or legal counsel of the parties hereto, by judicial or administrative process or otherwise by applicable law or regulation.

In full consideration of the provision of the services of the Administrator, the Company shall reimburse the Administrator for the costs and expenses incurred by the Administrator in performing its obligations and providing personnel and facilities hereunder. If requested to perform significant managerial assistance to portfolio companies of the Company, the Administrator shall be paid an additional amount based on the services provided, which amount shall not exceed the amount the Company receives from the portfolio companies for providing this assistance.

The Company shall bear all costs and expenses that are incurred in its operation and transactions and not specifically assumed by the Company’s investment adviser (the “Adviser”), pursuant to that certain Investment Advisory Agreement, dated as of December 4, 2012, by and between the Company and the Adviser. Costs and expenses to be borne by the Company shall include, but are not limited to, those relating to: (a) organization; (b) calculating the Company’s net asset value and net asset value per share (including the cost and expenses of any independent valuation firm); (c) fees and expenses, including travel expenses, incurred by the Adviser or payable to third parties in performing due diligence on prospective portfolio companies, monitoring the Company’s investments and, if necessary, enforcing the Company’s rights; (d) interest payable on debt, if any, incurred to finance the Company’s investments; (e) costs of offerings of the Company’s common stock and other securities; (f) the base management fee and any incentive fee; (g) distributions on the Company’s common stock; (h) administration fees payable to the Administrator under this Agreement; (i) transfer agent and custody fees and expenses; (j) the allocated costs incurred by the Administrator in providing managerial assistance to those portfolio companies that request it; (k) amounts payable to third parties relating to, or associated with, evaluating, making and disposing of investments; (l) brokerage fees and commissions; (m) registration fees; (n) listing fees; (o) taxes; (p) independent director fees and expenses; (1) costs associated with the Company’s reporting and compliance obligations under the Investment Company Act and applicable U.S. federal and state securities laws; (r) the costs of any reports, proxy statements or other notices to the Company’s stockholders, including printing costs; (s) costs of holding stockholder meetings; (t) the Company’s fidelity bond; (u) directors and officers/errors and omissions liability insurance, and any other insurance premiums; (v) litigation, indemnification and other non-recurring or extraordinary expenses; (w) direct costs and expenses of administration and operation, including audit and legal costs; (x) fees and expenses associated with marketing efforts, including deal sourcing and payments to financial sponsors; (y) dues, fees and charges of any trade association of which the Company is a member; and (z) all other expenses reasonably incurred by the Company or the Administrator in connection with administering the Company’s business, such as the allocable portion of overhead under this Agreement, including rent and the Company’s allocable portion of the costs and expenses of its chief compliance officer, chief financial officer, chief operating officer and their respective staffs. To the extent the Administrator outsources any of its functions, the Company shall pay the fees associated with such functions on a direct basis without profit to the Administrator.

The Administrator (and its officers, managers, partners, agents, employees, controlling persons, members, and any other person or entity affiliated with the Administrator, including without limitation its members) shall not be liable to the Company or its stockholders for any action by the Administrator (and its officers, managers, partners, agents, employees, controlling persons, members, and any other person or entity affiliated with the Administrator, including without limitation its members) in connection with the performance of any of its duties or obligations under this Agreement or otherwise as administrator for the Company, and the Company shall indemnify, defend and protect the Administrator (and its officers, managers, partners, agents, employees, controlling persons, members and any other person or entity affiliated with the Administrator, including without limitation the Adviser, each of whom shall be deemed a third party beneficiary hereof) (collectively, the “Indemnified Parties”) and hold them harmless from and against all damages, liabilities, costs and expenses (including reasonable attorneys’ fees and amounts reasonably paid in settlement) incurred by the Indemnified Parties in or by reason of any pending, threatened or completed action, suit, investigation or other proceeding (including an action or suit by or in the right of the Company or its security holders) arising out of or otherwise based upon the performance of any of the Administrator’s duties or obligations under this Agreement or otherwise as administrator for the Company. Notwithstanding the preceding sentence of this Paragraph 5 to the contrary, nothing contained herein shall protect or be deemed to protect the Indemnified Parties against or entitle or be deemed to entitle the Indemnified Parties to indemnification in respect of, any liability to the Company or its security holders to which the Indemnified Parties would otherwise be subject by reason of willful misfeasance, bad faith, gross negligence or reckless disregard of its obligations in the performance of the Administrator’s duties or by reason of the reckless disregard of the Administrator’s duties and obligations under this Agreement (to the extent applicable, as the same shall be determined in accordance with the Investment Company Act and any interpretations or guidance by the SEC or its staff thereunder).

The services of the Administrator to the Company are not to be deemed to be exclusive, and the Administrator and each affiliate are free to render services to others. It is understood that directors, officers, employees and stockholders of the Company are or may become interested in the Administrator and its affiliates, as directors, officers, members, managers, employees, partners, stockholders or otherwise, and that the Administrator and directors, officers, members, managers, employees, partners and stockholders of the Administrator and its affiliates are or may become similarly interested in the Company as stockholders or otherwise.

This Agreement shall become effective as of the date upon which the SEC declares the registration statement on Form N-2 to be effective, and shall remain in force with respect to the Company for two years thereafter, and thereafter continue from year to year, but only so long as such continuance is specifically approved at least annually by (i) the Board of Directors of the Company and (ii) a majority of those Directors who are not “interested persons” (as defined in the Investment Company Act) party to this Agreement.

This Agreement may be terminated at any time, without the payment of any penalty, by the Company, or by the Administrator, upon 60 days’ written notice to the other party. This Agreement may not be assigned by a party without the consent of the other party.

This Agreement may be amended pursuant to a written instrument by mutual consent of the parties.

This Agreement shall be construed in accordance with laws of the State of New York and the applicable provisions of the Investment Company Act, if any. To the extent that the applicable laws of the State of New York, or any of the provisions herein, conflict with applicable provisions of the Investment Company Act, the latter shall control.

This Agreement contains the entire agreement of the parties and supersedes all prior agreements, understandings and arrangements with respect to the subject matter hereof.

Any notice under this Agreement shall be given in writing, addressed and delivered or mailed, postage prepaid, to the other party at its principal office.

IN WITNESS WHEREOF, the parties hereto have caused this Agreement to be duly executed on the date above written.

[Signature Page to Administration Agreement]