Opinion for the Court filed by Circuit Judge EDWARDS.

In many criminal prosecutions, especially now in connection with cases growing out of narcotics trafficking, the Government is free to charge an ex-felon firearms count together with other counts, thereby permitting the jury to hear otherwise inadmissible evidence regarding a defendant’s prior conviction. This is such a case.

In United States v. Daniels, 770 F.2d 1111 (D.C.Cir.1985), we rejected a per se rule requiring separate trial of ex-felon counts, but we held that “sufficiently scrupulous regard,” id. at 1118, must be shown to protect the defendant from any undue prejudice resulting from joinder. In this case, the Government joined an ex-felon firearms count with three drug counts and resisted defense motions to sever, to introduce the defendant’s prior conviction by stipulation or to try the count to the judge. The Government also repeatedly referred to the defendant’s prior conviction during[*51] the trial, most notably through direct testimony of the defendant’s probation officer and cross-examination of the defendant’s brother. We hold that, on the facts presented, the high level of care necessary to prevent undue prejudice was not present in the trial of this case. Hence, we vacate Dockery’s conviction.

I. BackgRound

The Government charged appellant Johnny Dockery and his codefendant Edward Ramos ^[1] in a seven-count indictment arising out of four undercover drug purchases in August 1989. Counts one and two charged Ramos with distribution of crack cocaine on August 15 and 28. Count three charged Dockery with distribution on August 29. Counts four and five charged both Ramos and Dockery with distribution and possession with intent to distribute on August 30. Count six charged both defendants with using and carrying a firearm in relation to a drug trafficking offense. Count seven charged Dockery with possession of a firearm by an ex-felon, in violation of 18 U.S.C. § 922(g) (1988) (“count seven” or “the ex-felon count”).

The indictment arose from a series of undercover purchases made by Metropolitan Police Department (“MPD”) officer Jacqueline Middleton at 1616 E Street, N.E., Apt. 1. Middleton testified that, on August 15 and 28, she purchased crack cocaine from Ramos; Middleton stated that Dockery was present during these transactions, but that he handled neither the drugs nor the money. Trial Transcript (“Tr.”) vol. I, at 107-11. Subsequently, on August 29, Middleton walked down E Street and encountered Dockery, who asked her what she wanted. Dockery sent a juvenile into the apartment building with Middleton; the juvenile then went into the apartment while Middleton waited in the hall. The juvenile came out and said something to Dockery, who then went into the apartment and emerged with two ziploc bags each containing a $50 rock. Middleton took one and paid Dockery. Id. at 115. Finally, on August 30, Middleton approached Ramos and told him that she wanted “the usual.” Ramos took money from Middleton, then handed it to a juvenile, Hunter, who gave Middleton change. Hunter began “talking trash” with Middleton, and Dockery told Hunter to stop it. Id. at 118-19. Hunter had a brown pouch containing several zi-ploc bags and gave one containing a $50 rock to Middleton. Hunter continued to talk trash and Dockery grabbed him, threw him against the wall and told him to stop. Id. Based on her observations, Middleton testified that she thought Dockery was in control of the operation. Id. at 129.

After Middleton’s buy on August 30, the police executed a search warrant on the apartment and arrested Ramos, Hunter and a third individual. The police arrested Dockery in front of the apartment after the raid and search. MPD investigator Kirk Delpo testified that, when the police entered the apartment, Ramos and another person were on one side of the apartment and a third person, apparently Hunter, was on the other side of the apartment. Delpo testified that a gun was thrown to the floor from Hunter’s side of the apartment. Police later found a brown pouch containing several rocks of crack totaling approximately 15 grams in the apartment. Tr. vol. II, at 120-22, 134.

Prior to trial, Dockery sought to sever the ex-felon count. In the alternative, the defense offered to stipulate to Dockery’s prior conviction, stating that the judge could find Dockery guilty on count seven if the jury found him guilty of count six. The Government refused to accept either approach. The trial court denied Dockery’s motions, but ordered the indictment redacted to allege only the conviction of a prior felony — omitting any reference to the exact crime Dockery committed. The trial judge also refused to admit any testimony at trial that would have reflected the nature of the prior offense.

The Government presented Dockery’s prior felony conviction to the jury on five different occasions. The jury first learned that Dockery was an ex-felon when the[*52] indictment was read and the Government argued its opening. After summarizing the constructive possession count, the prosecutor stated,

And you’ll find out one more thing, Ladies and Gentlemen, that in this particular case, that Johnny Dockery, ..., is a convicted felon. And as a convicted felon, having constructive possession of that weapon, he incurs another offense.

Tr. vol. I, at 66. To prove the prior conviction, the Government called Dockery’s New York probation officer, who testified that Dockery was convicted of a felony in 1988. The Government also elicited testimony from the probation officer that Dockery was on probation at the time he was arrested in the case at hand, that the felony Dockery had been convicted of carried a possible prison term of greater than one year, and that Dockery was actually sentenced to five years’ probation. ^[2] The prosecutor also questioned the probation officer about Dockery’s conditions of probation certificate and his New York certificate of disposition. The Government attempted to move the two certificates into evidence, but the trial court ordered a stipulation in lieu of redacting the certificates. Tr. vol. II, at 105-08.

The Government next referred to Dock-ery’s conviction in cross-examining a key defense witness. In an attempt to impeach Middleton’s identification, the defense offered Dockery’s brother, who testified that Dockery had visited him on August 28 and early on August 29 at Fort Devens in Massachusetts. Allegedly in an effort to show the bias of Dockery’s brother, the Government referred on cross-examination to Dockery’s prior conviction. Defense counsel objected, to no avail, then the following colloquy ensued between the prosecutor and Dockery’s brother:

Q. You understand, Mr. Dockery, that why you’re here is because your brother is in some trouble. Is that right?

A. Yes, Ma’am.

Q. And you are here in an effort to assist him. Is that right?

A. Yes, Ma’am.

Q. And you are certainly aware that your brother had previously been convicted of an offense in New York. Is that right? ...

A. No, I don’t know anything about that, Ma’am.

Q. You didn’t know anything about his conviction in New York?

A. No, Ma’am.

Q. Okay.

And you didn’t know that he was on probation in New York?

A. No, Ma’am.

The Court: Excuse me, Ms. Vincent.

The Court: I would remind you, as I told you earlier, that that relates only to Mr. Dockery and has no connection with Mr. Ramos.

Q. So that you’re here now just to make sure that he doesn’t get into any trouble based upon his being arrested in Washington, D.C. Is that right?

A. Yes, Ma’am.

Tr. vol. IV, at 112-13. Finally, the Government referred to the prior conviction in its closing argument. Tr. vol. VI, at 58.

The defense also offered five witnesses who testified that the police had beaten and kicked the defendants after they had been handcuffed. It was uncontradicted that Ramos had to be taken to a hospital. The police uniformly denied any wrongdoing; defense counsel argued that the police persisted in their misidentification of Dockery to avoid liability for the beatings.

The jury found Dockery and Ramos guilty on the drug counts, but acquitted both defendants on the gun counts. Dock-ery timely noticed his appeal of the conviction. After conviction, Dockery’s counsel also moved for a new trial based on newly discovered evidence. The District Court, in a memorandum order, decided that no evi-dentiary hearing was necessary and denied[*53] the motion. Dockery again appealed and the two appeals were consolidated.

II. Analysis

A. Severance of the Ex-felon Count

Dockery contends that he was prejudiced by the trial court’s refusal to sever the ex-felon count because the jury was thereby made aware of his prior conviction. We find that, on the facts of this case, the refusal to sever was an abuse of discretion. When trying an ex-felon count together with other counts, the trial judge must “proceed with caution” to avoid undue prejudice. Daniels, 770 F.2d at 1118. Because that standard was not followed in this case, we vacate Dockery’s conviction.

Dockery does not contend that the counts were improperly joined under Rule 8 of the Federal Rules of Criminal Procedure. Fed.R.Crim.P. 8(a) (“Two or more offenses may be charged in the same indictment ... if the offenses charged ... are based on the same act or transaction_”). Rather, he moved before trial for severance under Rule 14:

If it appears that a defendant or the government is prejudiced by a joinder of offenses or of defendants in an indictment or information or by such joinder for trial together, the court may order an election or separate trials of counts, grant a severance of defendants or provide whatever other relief justice requires.

Fed.R.Crim.P. 14.

A defendant’s interest in avoiding introduction of prior crimes evidence is clear and compelling. “The exclusion of other crimes evidence is not simply a ‘technicality’ designed to prevent law enforcement personnel from doing their job; it reflects and gives meaning to the central precept of our system of criminal justice, the presumption of innocence.” Daniels, 770 F.2d at 1118. The primary concern is that prior crimes evidence “weigh[s] too much with the jury and ... overpersuade[s] them as to prejudge one with a bad general record and deny him a fair opportunity to defend against a particular charge.” Michelson v. United States, 335 U.S. 469, 476, 69 S.Ct. 213, 218, 93 L.Ed. 168 (1948). As we noted in Daniels, “There is ... a high risk of undue prejudice whenever ... joinder of counts allows evidence of other crimes to be introduced in a trial of charges with respect to which the evidence would otherwise be inadmissible.” 770 F.2d at 1116 (emphasis added).

Daniels, of course, fashioned no per se rule requiring the severance of ex-felon counts. Cf. United States v. Busic, 587 F.2d 577, 585 & n. 9 (3d Cir.1978) (suggesting that severance should be granted unless the government proves the prior crime evidence is independently admissible), rev’d on other grounds, 446 U.S. 398, 100 S.Ct. 1747, 64 L.Ed.2d 381 (1980). ^[3] Nevertheless, the court was clear that the utmost care is necessary in conducting a joined trial that includes an ex-felon count.

[W]e are unable to say that joinder in such a situation is always an abuse of discretion under Rule 14_ In particular, we think the district court in this case demonstrated a sufficiently scrupulous regard for the defendant’s right to a fair trial to warrant affirmance; on these facts and this record, we are not convinced that the joinder of the ex-felon offense caused Daniels undue prejudice. We emphasize, however, that joinder decisions must be informed by a respect for the special problems created by the introduction of other crimes evidence, and that it consequently will behoove prosecutors and trial judges to proceed with caution when situations similar to this one face them in the future.

[*54] Our affirmance of this conviction is closely tied to the facts at hand and the scrupulous care taken by the trial judge. In general, it is neither commendable nor prudent for prosecutors to seek to use ex-felon counts to escape the encumbrances of the presumption of innocence.

770 F.2d at 1118-19.

Daniels thus both constrains the trial judge’s discretion in denying motions to sever ex-felon counts and informs the review we make of the trial court’s decision. While we review a trial court’s Rule 14 decision for abuse of discretion, see Daniels, 770 F.2d at 1115, the law of the circuit plainly limits that discretion. Abuse of discretion review means “that the court has a range of choice, and that its decision will not be disturbed as long as it stays within that range and is not influenced by any mistake of law.” Kern v. TXO Production Corp., 738 F.2d 968, 970 (8th Cir.1984) (emphasis added). Cf. Cupp v. Naughten, 414 U.S. 141, 146, 94 S.Ct. 396, 400, 38 L.Ed.2d 368 (1973) (“[An] appellate court will, of course, require the trial court to conform to constitutional mandates, but may likewise require it to follow procedures deemed desirable from the viewpoint of sound judicial practice_”).

On the facts of this case, we are left with the firm conviction that “[t]he proper balance between judicial economy and the prejudicial effect of evidence of prior convictions was not struck in this instance.” Panzavecchia v. Wainwright, 658 F.2d 337, 341 (5th Cir. Unit B Oct. 1981) (granting writ of habeas corpus; trial infected by ex-felon counts). In Daniels, we enumerated the safeguards the trial judge employed in conducting the joined trial that included an ex-felon count. Comparable safeguards were not followed in this case. Although we are not creating a talismanic list of procedures, Daniels, and similar cases from other circuits, demonstrate how a trial judge can protect the defendant’s fair trial right while achieving the time savings of a joined proceeding.

For example, in Daniels, the prior conviction was introduced by stipulation. 770 F.2d at 1114; see also Burgess, 791 F.2d at 679. Here, the Government insisted on offering its proof. Although the Government need not accept a proposed stipulation, see United States v. Weisz, 718 F.2d 413, 431 n. 106 (D.C.Cir.1983), cert. denied, 465 U.S. 1027, 104 S.Ct. 1285, 79 L.Ed.2d 688 (1984); United States v. Cockerham, 476 F.2d 542, 545 (D.C.Cir.1973) (per curiam) (defendant has no right to stipulate inflammatory facts), recognizing this prerogative does not solve the problem in this case. The Government’s desire to offer live testimony simply factors into whether the count should be severed for undue prejudice. Unlike Weisz and Cock-erham, where the defendant sought to stipulate to the most damning elements of the offense charged, the problem here is the best way to avoid prejudice flowing from the element of one count to another. The Government has no “right” to both its proof and a joined trial.

Additionally, the Government’s right to introduce its proof is always subject to the trial court’s responsibility under Fed.R.Evid. 403 to limit unduly prejudicial or cumulative evidence. See United States v. O’Shea, 724 F.2d 1514, 1516 (11th Cir.1984) (in Rule 403 balancing, proffered stipulation reduces probative value of Government’s evidence); United States v. Bass, 794 F.2d 1305, 1312 n. 6 (8th Cir.) (“a proper rule 403 balancing analysis will incorporate some assessment of the need for the allegedly prejudicial information in light of a valid stipulation”), cert. denied, 479 U.S. 869, 107 S.Ct. 233, 93 L.Ed.2d 159 (1986); United States v. Ellison, 793 F.2d 942, 949 (8th Cir.) (“ ‘The government is not bound to stipulate to such facts unless the prejudicial aspects of the testimony in context outweigh its probative value....’”) (quoting United States v. DeJohn, 638 F.2d 1048, 1053 (7th Cir.1981)), cert. denied, 479 U.S. 937, 107 S.Ct. 415, 93 L.Ed.2d 366 (1986).

The Government also argues that the nature of the proposed stipulation intruded on the factfinder’s province. The defense at least initially sought to stipulate to the facts outside the presence of the jury. In[*55] this, the defense sought an even more protective device than that employed in Daniels and Burgess. In'essence, the defense sought a bench trial on count seven. Under Fed.R.Crim.P. 23(a), the Government may veto the defendant’s request for a bench trial, so long as that veto does not otherwise violate the defendant’s constitutional rights. See Singer v. United States, 380 U.S. 24, 37, 85 S.Ct. 783, 791, 13 L.Ed.2d 630 (1965) (“Rule 23(a) does not require that the Government articulate its reasons for demanding a jury trial at the time it refuses to consent to a defendant’s proffered waiver.”); United States v. Moon, 718 F.2d 1210, 1217 (2d Cir.1983) (“Nothing in the [Sixth Amendment] guarantees one the right to select his own tribunal or the right to a speedy and public trial by a fair and impartial judge.”), cert. denied, 466 U.S. 971, 104 S.Ct. 2344, 80 L.Ed.2d 818 (1984). Nonetheless, as in the case of stipulations, the Government has no right to a joined jury trial.

The Government also objects that the proposed stipulation would have eliminated the possibility that the judge and the jury might have come to differing conclusions on counts six and seven. This argument has some force because the “uses or carries” prong of count six, see 18 U.S.C. § 924(c) (1988), is somewhat different than the “possession” prong of count seven. See United States v. Bruce, 939 F.2d 1053, 1054-56 (D.C.Cir.1991) (“uses or carries” requires some showing that possession of the gun relates to the narcotics violation). Still, one proper resolution, and one clearly encompassed by Dockery’s request, see Motions Tr. at 15, would have been to instruct the jury on both “possession” and on “carrying and use” and then have the judge apply the ex-felon count in sentencing should the jury find possession. ^[4] Thus, no possible result would have been foreclosed. While perhaps justified in opposing the defense’s proposed alternatives, the Government bears some burden to propose alternatives sufficient to protect the defendant. See Berger v. United States, 295 U.S. 78, 88, 55 S.Ct. 629, 633, 79 L.Ed. 1314 (1935) (In addition to pursuing convictions, United States Attorney must seek a fair trial.). Similarly, the trial court, in exercise of its continuing authority to assure a fair trial, should have sought alternative means to protect both parties or forced the Government to choose between its proof and the joint trial.

The Daniels court also noted the use of jury instructions in that case, although it was “not ... sanguine concerning [their] efficacy.” 770 F.2d at 1118; see also Silva, 745 F.2d at 844 (noting instructions as a means to mitigate prejudice from joined ex-felon counts); Yates, 734 F.2d at 370 (same); Valentine, 706 F.2d at 290 n. 7 (same); Panzavecchia, 658 F.2d at 341 (same). Here, the court offered no instructions cautioning the jury about inferring propensity to commit the drug crimes. When Dockery’s probation officer testified, the trial judge reread the indictment, and then stated, “The evidence that you just heard with respect to a prior felony conviction is admitted solely for the purpose of meeting that one element, that he had been convicted before. It should not be considered by you with respect to whether he is guilty or not guilty of count seven.” Tr. vol. II, at 109. This comment, however, did not address the probability that the prior conviction would bias the jury to defendant’s prejudice on the other counts. During the Government’s cross-examination of Dockery's brother, the trial judge again injected a word of caution: “I would remind you, as I told you earlier, that that relates only to Mr. Dockery and has no connection with Mr. Ramos.” Tr. vol. IV,[*56] at 112. This comment similarly did not address the prejudice problem. During the final charge, the trial court instructed the jury that the parties had stipulated to Dockery’s prior conviction, Tr. vol. VII, at 26, but the trial judge again failed to caution the jury against possible prejudice.

Dockery’s counsel may have been remiss in not requesting more focused cautionary instructions, for the trial court bears no burden to offer cautionary instructions sua sponte each time prior crimes evidence is introduced. United States v. Miller, 895 F.2d 1431, 1438-39 (D.C.Cir.), cert. denied, — U.S. -, 111 S.Ct. 79, 112 L.Ed.2d 52 (1990). But cf. United States v. Lewis, 693 F.2d 189, 196—97 (D.C.Cir.1982) (when evidence is highly prejudicial, trial judge must sua sponte offer defense counsel opportunity to waive instruction or have an instruction given). Nonetheless, defense counsel had entered repeated and continuing objections to the prior crimes evidence, and the trial court had a continuing obligation to assure a fair trial. Appropriate instructions are one way the trial judge might have exercised, with “sufficiently scrupulous regard” for the defendant’s rights, his discretion to conduct a joined trial. See Daniels, 770 F.2d at 1118.

Perhaps most significantly, the conviction was emphasized to the jury much more here than in Daniels. Here, the Government or the trial court adverted to the conviction on six separate occasions, most notably in the Government’s direct examination of the probation officer and in its cross-examination of Dockery’s brother, see section II.B infra. In both instances, the prosecutor introduced both Dockery’s prior conviction and that he was on probation at the time he was arrested. This last fact was utterly irrelevant to the trial. ^[5]

Finally, the evidence supporting the ex-felon count was extraordinarily weak. The Government tried the count on a constructive possession theory — that Dockery, who was not in the apartment at the time of the raid, nevertheless controlled Hunter, who allegedly threw the gun to the ground during the search. The jury acquitted Dock-ery of the gun counts; we doubt that the prosecution, if put to a separate trial, would have pursued the charges.

We are mindful that joined trials may, in appropriate circumstances, conserve judicial resources. To that end, the Government and trial courts may exercise their discretion to try ex-felon counts with other counts in a single trial. But, when doing so, the Government and the trial court must avoid undue prejudice to defendants. Here, as in Daniels, we do not limit the means by which a trial judge can endeavor to meet both goals. However, as we made clear in Daniels, if Rule 14 and the presumption of innocence are to be given their proper content, a joint trial of ex-felon counts must be conducted with “sufficiently scrupulous regard for the defendant’s right to a fair trial.” Daniels, 770 F.2d at 1118. In this case, adequate precautions were not taken and we therefore find an abuse of discretion.

B. Government Cross-Examination of Dockery’s Brother

The defense introduced Dockery’s brother to impeach Officer Middleton’s identification. The Government cross-examined him, as quoted above, by referring to Dockery’s prior conviction and probation. Defense counsel objected to this line of inquiry. We agree that it was error for the trial judge to allow the prosecutor to pursue this cross-examination.

The trial judge thought the cross-examination permissible given that the jury had already been exposed to the prior conviction. Tr. vol. IV, at 108-09. The trial court also thought the jury should be able to assess the “totality of the circum[*57] stances” because of the conflict between the witnesses to the alleged beatings and the officers’ denials. Id. at 108.

We have held above that, if not severed, the joint trial of an ex-felon count must be sanitized. Hence, the reference here by the prosecution would not be harmless. Furthermore, this type of cross-examination should have been excluded under Fed.R.Evid. 403. It is highly prejudicial and, as Government counsel essentially admitted at oral argument, completely without probative value. ^[6]

Therefore, there is no merit to the Government’s claim that Dockery suffered no undue prejudice from the introduction of the ex-felon evidence because it would have come in in any event through the cross-examination of his brother. As we have said, the cross-examination itself was beyond the bounds of permissible inquiry, so the failure to sever was hardly made harmless by the cross-examination.

C. New Trial Request

Because we vacate Dockery’s conviction, we do not reach the merits of his motion based on the newly-discovered evidence. On remand, the District Court may exercise its usual discretion over the proffered testimony.

III. CONCLUSION

For the foregoing reasons, we hold that, on the facts of this case, the trial court erred in permitting, over objection, the joint trial of an ex-felon count. Additionally, the trial judge erred in permitting the Government to use the prior conviction in cross-examining a key defense witness. We therefore vacate the convictions and remand to the District Court.