SUMMARY ORDER

Plaintiff-Appellant Christine Larkins appeals from a final judgment of the district court affirming the decision of an administrative law judge (ALJ) to deny her Social Security disability benefits. Larkins had been employed at Bell Atlantic (or its predecessors) for 32 continuous years when she stopped working on June *19415, 1995, due to vision problems. She was later diagnosed with multiple sclerosis. Her symptoms include chronic fatigue, unsteady gait, and leg weakness. After filing for disability benefits on April 2, 1997, she received a hearing before an ALJ on June 22, 1998. On August 17, 1998, the ALJ determined that Larkins was not disabled. The Appeals Council denied review, but later vacated that order so that it could consider additional reports by Larkins’ doctors. Upon reconsideration, the Appeals Council again denied review on October 31, 2001. On May 1, 2003, the district court (Johnson, J.) affirmed the ALJ’s decision. Larkins then appealed to this court.

We may only set aside an ALJ’s decision if it is based upon legal error or is not supported by substantial evidence. See Shaw v. Chater, 221 F.3d 126, 131 (2d Cir. 2000). After carefully reviewing the administrative record in this case, we vacate and remand the district court’s decision with directions to remand to the ALJ for reconsideration and further fact-finding in accordance with this opinion.

Larkins challenges the ALJ’s reasoning at steps three and four in the five-step process used for determining whether a claimant is “disabled” under 42 U.S.C. § 423(d)(1)(A). See Green-Younger v. Barnhart, 335 F.3d 99, 106 (2d Cir. 2003) (describing the five-step decision process and observing that the claimant bears the burden of proof with respect to the first four steps). In reaching his decision, the ALJ had before him the following evidence: 1) various reports by Larkins’ treating physician, Dr. Bagley, a board-certified neurologist; 2) a report by Dr. Kaye, a non-examining government consultant; and 3) Larkins’ testimony at the hearing. After the ALJ’s decision, Larkins submitted to the Appeals Council additional reports and letters-three from Dr. Bagley and one from examining neurologist Dr. Rabinovici. To the extent that this new evidence bears on Larkin’s medical condition prior to the ALJ’s decision, we review the ALJ’s decision as though this evidence was before him. See Perez v. Chater, 77 F.3d 41, 45 (2d Cir. 1996).

We hold that the ALJ lacked an adequate basis for determining, at step three, that Larkins did not have a listed impairment. The ALJ gave no specific reasons for his conclusion that she did not have a listed impairment, but stated only that “[n]o treating or examining physician has mentioned findings equivalent in severity to the criteria of any listed impairment,” and that he had paid “[pjarticular attention” to Listing 11.09. Listing 11.09 provides that multiple sclerosis, when accompanied by one of three other conditions, qualifies as a per se disability. See 20 C.F.R. 404, Subpt. P, App. 1, § 11.09. Larkins argues that her multiple sclerosis is accompanied by one of these three conditions, namely, “[disorganization of motor function,” which is defined by a cross-reference to section 11.04B as “[significant and persistent disorganization of motor function in two extremities, resulting in sustained disturbance of gross and dexterous movements, or gait and station....” See Listing 11.04B. Section 11.04B, in turn, refers to section 11.00C, which states: “Persistent disorganization of motor function in the form of paresis or paralysis, tremor, or other involuntary movements, ataxia and sensory disturbances (any or all of which may be due to cerebral cerbellar, brain stem, spinal cord, or peripheral nerve dysfunction) which occur singly or in various combinations, frequently provides the sole or partial basis for decision in cases of neurological impairment. The assessment of impairment depends on the degree of interference with locomotion and/or interference with the use of fingers, hands, and arms.”

*195We conclude that the administrative record was simply not developed to a sufficient degree to permit the ALJ to determine whether Larkins suffers from such a “disorganization of motor function.” There was little, if any, medical evidence that spoke directly to the terms used in Listings 11.04B and 11.00C. And, it is well established that an ALJ has an affirmative duty to seek out information to fill any clear gaps in the administrative record, regardless of whether the claimant is represented by counsel. See Shaw, 221 F.3d at 134; Rosa v. Callahan, 168 F.3d 72, 79 (2d Cir. 1999). On remand, the ALJ should develop the record accordingly and consider anew whether Larkins’ condition meets the requirements of 11.04B.

If the ALJ should determine, on remand, that Larkins had a listed impairment at the relevant time, the inquiry is over and the only task that remains is the calculation of disability benefits. If, however, the ALJ determines that Larkins did not have a listed impairment, he should then reconsider his earlier determination, at step four, that Larkins retained the “residual functional capacity” to perform one of her past jobs, namely that of a “circuit designer” at a phone company. In doing this, the ALJ should especially seek to resolve seeming inconsistencies between Dr. Bagley’s March 4, 1998 report, and his earlier and later reports together with Dr. Rabinovici’s 1999 report.

We therefore VACATE and REMAND the district court’s judgment with directions to REMAND to the ALJ for reconsideration and further fact-finding in accordance with this opinion.