Jon Barkley appeals a summary judgment on his hostile work environment and retaliation claims against his employer, Singing River Electric Power Association (“SREPA”), under 42 U.S.C. § 1981. We affirm.

I.

Barkley worked at SREPA for nearly eleven years starting in 1997 as a meter reader. He alleges that during that time, his coworkers called him names such as “n* * * *r” and “black gorilla” on a nearly daily basis and made references to “shackles” and “plantations.” It is unclear how many of those comments were made in Barkley’s presence or were even about him, because he only vaguely described the alleged harassment1 and said that he heard most of the racial slurs second-hand from a hidden tape recorder he left around his coworkers.

Barkley reported the racial slurs only to his immediate supervisor and friend, Ken Papania. In 2001, Barkley complained to Papania about one specific instance, a coworker, Danny Dillard, calling him a “n* * * *r” during an argument. Papania notified the General Supervisor, Lee Hedegaard, about Barkley’s complaint, and Hedegaard spoke to Barkley and Dillard. During the meeting, Barkley admitted that he had started the altercation by making disparaging remarks about Dillard’s daughter and that Dillard did not actually call him the offensive word during the argument but had used the term months earlier. Regardless of that, Hedegaard told them he would not tolerate such language.

Barkley complained to Papania a second time, seven years later in January 2008, about his coworkers using the term “n* * * *r.”2 Papania told Barkley that he should be used to hearing that word, “because blacks used the term so much,” and that his coworkers “were just a ‘bunch of rednecks’ [so] he should not be offended.”3 Barkley also vaguely asserts that he told Papania about the racial slurs “throughout my whole years of really being there, really,” but he does not support *256that broad statement with any other specific examples. Barkley claims he told other people about the discrimination, but he never deposed those individuals, nor did they submit affidavits.

In April 2008, Barkley informed Papania that he intended to resign on July 23 to spend time with his wife, who was pregnant with triplets in a high-risk pregnancy. Although Barkley signed an acknowledgment of resignation on April 4, he claims that ten minutes later, he called Hedegaard to rescind the resignation. Hedegaard denies that he spoke to Barkley, and although Barkley claims that in the ten minutes between signing the resignation and calling Hedegaard, he also told a friend and another employee that he wanted to rescind his resignation, he has not provided their testimony.

On April 25, Barkley called Annette Riley, the Human Resources Manager, to discuss rescinding his resignation.4 Riley denies that Barkley told her he wanted to rescind his resignation; instead, she claims that Barkley asked to change his resignation date to July 25. Riley provided a copy of Barkley’s signed resignation form on which she wrote, “Mr. Barkley called at 5:22 p.m. and changed the date to July 25.” Riley signed and dated those comments April 15, and Barkley has not refuted the document’s authenticity.

Barkley’s last day of work was June 8. He testified that he finished early because of changes in his wife’s condition. Before he left SREPA on his last day, he met with Papania and recorded the conversation.5 He did not discuss wanting to rescind his resignation. Instead, he told Papania he had another job offer. Papania told Barkley numerous times that he was sorry to see him go, and they talked at length about Barkley’s children and future plans. Barkley told Papania about the tapes he had made and that he had sent them to various national news organizations. Papania said that he did not understand what Barkley was talking about and that he was shocked.

II.

After leaving SREPA, Barkley sued it for violations of title VII and § 1981 alleging that he was subjected to a hostile work environment and that SREPA fired him in retaliation for complaining about racial discrimination. The district court granted SREPA’s motion for summary judgment on the title VII claims, because they were filed with the EEOC more than 180 days after Barkley’s employment had ended.6 The court also granted SREPA summary judgment on Barkley’s § 1981 claims. Barkley appeals on only the § 1981 claims.

III.

We review a summary judgment de novo. Bolton v. City of Dallas, 472 F.3d 261, 263 (5th Cir. 2006). Summary judgment is appropriate if there is no genuine issue of material fact and the moving party is entitled to judgment as a matter of law. Fed.R.Civ.P. 56(a); Anderson v. Liberty Lobby, Inc., 477 U.S. 242, 247, 106 S.Ct. *2572505, 91 L.Ed.2d 202 (1986). We view the evidence in the light most favorable to the non-moving party, Matsushita Elec. Indus. Co. v. Zenith Radio Corp., 475 U.S. 574, 587, 106 S.Ct. 1348, 89 L.Ed.2d 538 (1986), but once the moving party has carried its burden, the non-movant must come forward with specific facts showing a genuine factual issue for trial. Id. Conclusional allegations and denials, speculation, and unsupported assertions are insufficient to avoid summary judgment. SEC v. Recile, 10 F.3d 1093, 1097 (5th Cir. 1993).

A.

To prevail on a hostile work environment claim under § 1981, Barkley must show that (1) he belongs to a protected group, (2) he was subjected to unwelcome harassment, (3) the harassment complained of was based on race, (4) the harassment complained of affected a term, condition, or privilege of his employment, and (5) the employer knew or should have known of the harassment and failed to take prompt remedial action. See Ramsey v. Henderson, 286 F.3d 264, 268 (5th Cir. 2002) (in the title VII context).7 “A hostile work environment exists ‘when the workplace is permeated with discriminatory intimidation, ridicule, and insult, that is sufficiently severe or pervasive to alter the eonditions of the victim’s employment and create an abusive working environment.’ ” Stewart v. Miss. Transp. Comm’n, 586 F.3d 321, 328 (5th Cir. 2009) (quoting Nat’l R.R. Passenger Corp. v. Morgan, 536 U.S. 101, 116, 122 S.Ct. 2061, 153 L.Ed.2d 106 (2002)). Looking at the totality of the circumstances, id., we consider “the frequency of the discriminatory conduct; its severity; whether it is physically threatening or humiliating, or a mere offensive utterance; and whether it reasonably interferes with an employee’s work performance.” Harvill v. Westward Commc’ns, LLC, 433 F.3d 428, 434 (5th Cir. 2005). The working environment must be both “objectively offensive, meaning that a reasonable person would find it hostile and abusive, and subjectively offensive, meaning that the victim perceived it to be so.” Shepherd v. Comptroller of Pub. Accounts, 168 F.3d 871, 874 (5th Cir. 1999).

SREPA asserts that Barkley has not raised a question of fact as to whether the discrimination was severe or pervasive. It characterizes Barkley’s allegations as “vague, confusing, and contradictory”8 and notes that he described much of the discrimination in broad terms followed by, as he put it, “blah, blah, blah” or “whatever, whatever.” See supra note 1. Even *258though Barkley claims that the harassment occurred frequently and that he both reported it to other employees and secretly recorded it, SREPA notes that he has not pointed to any evidence in the record to corroborate his testimony.

Barkley does not respond to SREPA’s point regarding the lack of corroboration. He seemingly believes that his affidavit and deposition testimony should be sufficient. For some hostile work environment claims, a plaintiffs sworn testimony may be enough to raise a fact issue. Here, however, the allegations are ambiguous and generalized. “[T]o defeat a motion for summary judgment, the nonmoving party must go beyond the pleadings and by her own affidavits, or by the depositions, ... designate specific facts showing that there is a genuine issue for trial.” Cotroneo v. Shaw Env’t & Infrastructure, Inc., 639 F.3d 186, 191 (5th Cir. 2011) (emphasis added) (internal quotation marks omitted).

In Ramsey, 286 F.3d at 269, the plaintiff made similarly vague and generalized allegations of racial harassment. We declined to find that the plaintiff had raised a fact issue regarding whether she suffered racial discrimination:

The record is rife with vague assertions of [over thirteen years of] racial animus.... However, other than assertions that her supervisor ... discriminated against her for dating an African American male, there are no specific allegations of racial discrimination against any other employees. [The plaintiff] alleges that she ‘suffered ongoing racial harassment from black females,’ but points to no concrete examples. [She] explains that the harassment increased when she began dating an African American and subsequently had a child with him but again gives no concrete examples beyond mere conclusory assertions. While claiming that the racial harassment became extreme after beginning work under [a new supervisor], the only example she cites is reference to a remark where [the supervisor] made a derogatory comment about [her]. However, this statement was not heard by [the plaintiff] nor does [she] establish to whom the remark was made. This Court has cautioned that conclusory allegations, speculation, and unsubstantiated assertions are inadequate to satisfy the nonmovant’s burden in a motion for summary judgment.

Id. (internal quotation marks omitted). For the same reasons, Barkley’s generalized allegations cannot defeat summary judgment. At best, they might show that the harassment occurred frequently, but Barkley does not describe the severity of the harassment, nor does he allege that the harassment was physically threatening or humiliating. Although he names several of his alleged harassers in his deposition and states which specific epithets he was called, the facts are such a cobbled mess that it is difficult to discern more than one or two specific instances of harassment.

Further, Barkley testified that the harassment had no impact on his work performance. He stated, “[W]hen I heard it, it doesn’t really affect me like it effects some people. Some people get in an uproar to hear it. With me, you hear it, you just go about your business. I’m not one of those people who get upset when they hear whatever, whatever.” Actionable harassment “will detract from employees’ job performance, discourage employees from remaining on the job, or keep them from advancing in their careers.” Harris v. Forklift Systems, Inc., 510 U.S. 17, 22, 114 S.Ct. 367, 126 L.Ed.2d 295 (1993). Barkley worked at SREPA for'nearly eleven years and never stated that his coworker’s racist language detracted from his job *259performance, discouraged him from remaining on the job, or hindered his career advancement. His reason for leaving SREPA was to be with his pregnant wife, not to get away from racial slurs.

In light of the totality of the circumstances and the vagueness of the allegations, Barkley has failed to raise a genuine dispute as to whether the harassment was severe and pervasive. Thus, we affirm summary judgment on the hostile work environment claim.

But even assuming that the harassment was severe and pervasive, Barkley has not sufficiently shown that there is a question whether his supervisors knew of the harassment and failed to take remedial action. Barkley states that the only supervisor he complained to was Papania. When Barkley told Papania about his altercation with Dillard in 2001, Papania immediately informed his own supervisor, Hedegaard, about the harassment. Hedegaard met with Barkley and Dillard and told them he would not tolerate racial slurs in the workplace. Barkley claims that Hedegaard’s response was ineffective and that he told Papania about the continuing racial slurs numerous times after that incident. The test, however, is not whether the harassment stopped but whether the action taken by the employer was “reasonably calculated to end the harassment.” Stewart, 586 F.3d at 329. Considering Barkley’s complaint — that only a single coworker called him a racial epithet — Papania’s and Hedegaard’s responses were reasonably calculated to end the harassment they knew of.

Barkley makes only vague allegations of further reporting to Papania, describing a single complaint seven years after his first complaint. Barkley claims that he told Papania in January 2008 about “racial harassment,” but “Papania refused to take any action, and informed me that I should just be used to the term ‘n* * * *r’ because blacks used the term so much.” But Barkley appears to rest on the apparent offensiveness of Papania’s comments and does not address the effectiveness of whatever response Papania took. Barkley’s conclusional allegation that Papania did not take any action cannot hold up when he does not even allege that the harassment continued after that meeting. Thus, even if the harassment was severe and pervasive, SREPA has shown that it took reasonable remedial actions to end the harassment it knew of.

B.

We utilize a burden-shifting framework to analyze claims of retaliation where there is no direct evidence. Byers v. Dallas Morning News, Inc., 209 F.3d 419, 427 (5th Cir. 2000). The plaintiff must first establish the three elements of a prima facie case: (1) He engaged in a protected activity, (2) he suffered an adverse employment action, and (3) a causal link exists between the protected activity and the adverse action. See Gee v. Principi, 289 F.3d 342, 345 (5th Cir. 2002) (in the similar Title VII context). If the plaintiff can establish a prima facie case, the burden shifts to the defendant to demonstrate a legitimate nondiscriminatory reason for its action. Id. If the employer meets its burden, then the defendant cannot withstand summary judgment unless he raises a genuine issue of material fact as to whether the employer’s stated reason was merely a pretext for the real, discriminatory purpose. Id.

Assuming that Barkley complained about the racial harassment and that he was prohibited from rescinding his resignation, he has failed to show the existence of a causal link and thus cannot establish a prima facie case. Barkley claims that Hedegaard did not allow him to withdraw his *260resignation in retaliation for complaining about racial harassment. His only support for that conclusional assertion is the temporal proximity between his January 2008 complaint to Papania and his inability to rescind his resignation in April 2008.

Barkley fails to show, however, that Hedegaard, the person who he claims denied his request, knew of his January complaint to Papania. Hedegaard knew of Barkley’s 2001 complaint, but that was seven years earlier. And Papania may have informed Hedegaard of Barkley’s complaint in 2008 as he did in 2001, but there is no evidence of that in the record. Without knowledge of Barkley’s January 2008 involvement in protected activity, Hedegaard could not have retaliated against Barkley by prohibiting him from withdrawing his resignation. See Manning v. Chevron Chem. Co., 332 F.3d 874, 883 n. 6 (5th Cir. 2003). Thus we affirm summary judgment on Barkley’s retaliation claim.

Additionally, according to this circuit’s caselaw, even if Hedegaard did know of Barkley’s January complaint, the four-month gap in time, standing alone, is insufficient to establish prima fade evidence of causation.9 Fifth Circuit precedent on that point, although largely unpublished, fits with the Supreme Court’s requirement that the temporal proximity be “very close” to show causation. Clark Cnty. Sch. Dist. v. Breeden, 532 U.S. 268, 273-74, 121 S.Ct. 1508, 149 L.Ed.2d 509 (2001).10 Barkley cites several cases that he claims state that a four-month period is sufficient, but those cases say no such thing.11

Finally, even if we assume that Barkley could make a prima fade showing of retaliation, SREPA has proffered a legitimate, nondiscriminatory reason for why it would not have allowed Barkley, or any employee, to withdraw his resignation: Once a resignation is signed, the company immediately moves to fill the vacancy, and if an employee wanted to withdraw a resignation after the slot was filled, it could create problems for SREPA. Barkley argues that SREPA’s explanation is pretextual merely because Hedegaard testified to it and it is not a written company policy, but he does not cite any cases in which we have said that such a fact should make a difference. Nor has Barkley proffered the names of any employees who were allowed to rescind their resignation to show that the policy was not applied uniformly. Thus, Barkley has failed to raise a genuine dispute of fact regarding his retaliation claim.

AFFIRMED.