VoxNexus / Research

TEXT-TO-SPEECH · VOICE CLONING

RSE-TTS

Compact Text-to-Speech through Shared Text-Aware
Reference Encoding and Generation

Hear the voices ↓Read the paper ↗
Ming Ma Mingjie Chen Yuanzhong Zheng Hengyao Bao Hailin Yu
Shijing Yu Hao Qiu Gongfan Cao Xiao Li Xuemin Zhao Youhai Jiang

VoxNexus

01 / THE PAPER

Abstract

Can a speech generator also serve as an effective voice encoder? We show that a pretrained autoregressive text-to-speech (TTS) backbone can be repurposed for reference compression, improving speaker similarity in subsequent synthesis. Jointly encoding reference speech and its transcript further strengthens this capability. Controlled ablations and activation interventions trace the text benefit through intermediate acoustic representations into the compressed vectors that condition generation. Building on these findings, we introduce RSE-TTS, whose Reconstructive Speaker Encoder (RSE) shares the generation backbone and compresses paired reference text and speech into eight continuous vectors. Sharing reduces encoder–generator parameters by 48% relative to independent pretrained encoding. Large-scale English training yields a 118M-parameter generator that improves both word error rate and speaker similarity over the corresponding official MOSS-TTS-Nano cloning modes on all three English benchmarks. Combined with acoustic continuation, RSE-TTS approaches the speaker similarity of substantially larger systems while achieving lower WER than the evaluated CosyVoice 3 and MOSS-TTS-Local models.

RSE-TTS architecture: a shared Global Transformer encodes reference text and speech into eight query vectors; the generation pass consumes the compact condition and target text, with Local AR and codec decoding producing speech.
One shared backbone for reference encoding and speech generation.

02 / RESULTS

Compact model.
Competitive quality.

Speech accuracy and voice similarity,
at a fraction of the model size.

Comparison of model generation-chain parameters against WER and speaker similarity across three English benchmarks.
Averages across Seed-TTS, CV3-eval, and MiniMax. Model size includes the generator and waveform decoder.

03 / CUSTOM VOICES

Custom voices.

Explore expressive styles and four curated voices.

Stylized voices

Conversations, storytelling, and expressive delivery.

Conversation02

What feels comfortable

11.0s

Yeah, I used to think lighter meant faster, too. Then I tried it on a longer run and, honestly, I just felt tired. So now I'm a little more interested in what actually feels comfortable.

A coffee instead

9.2s

Wait, so do you make it fresh every time? I mean, that sounds great, but I'd probably get halfway through and just order a coffee instead. How do you actually keep up with it?

Podcast & hosting02

A little holiday conversation

28.6s

Whoa! Hello, everybody. It's me, Sunnywhisk. Thank you so much for watching this new sketch in, well, Christmas kind of theme. So let me know what is your opinion on Christmas and the holidays and everything. I can kind of guess it. That's kind of what this video was about, but I still want to know. Maybe you don't even celebrate Christmas. Tell me what do you think about all of these holiday kind of things. And if you're new here, I would love to have you subscribe. I post sketches like this on Wednesday and on a Saturday, I post stories or opinions or getting to know each other a little bit better videos like that.

More than a paycheck

18.5s

Most people don't buy something bigger than a house, so it's it's important to care about your clients and care about doing good for them more so than just thinking it's going to be a quick, easy paycheck. Because that is definitely not how you could describe real estate.

Storytelling02

The long game

15.4s

Pretending is weakness. Growth begins with humility. Six, you play the long game. You value what's timeless, not what's trending. True intelligence isn't loud. It's grounded. It's calm. It's wise.

A place to slow down

18.2s

Picture sunlight streaming through terracotta-tiled walkways, illuminating verdant foliage and casting shifting shadows across rustic wooden beams. This is more than just a design statement; it's an invitation to experience a new way of living, where each corner offers a fresh perspective.

Sports02

Once the game starts

9.0s

Did you notice the crowd at all while you were playing? Because from where I was sitting, it felt pretty loud. Or do you sort of switch all that off once the game starts?

All eyes on you

15.1s

I think obviously the amount of eyeballs, specifically the two games at US Bank Stadium, the Lakeville South game and the Edina game. You played great in both of those. That did you get a sense there were more people watching, not necessarily watching the football, they were watching you.

Explainers02

An unusual detail

25.0s

Next up is yellow eyes. Yellow is not the kind of color you would assume anyone's eyes to be, but despite what you may believe, yellow eyes are a real possibility and are very rare. Yellow eyes are often called tiger eyes, as tigers actually have yellow eyes. However, some humans have them too, and it's not a good thing. Those with yellow eyes sometimes actually have kidney problems. That's right, guys.

Inside the game

28.4s

The end result of the effect is usually difficult to distinguish from the end result of tessellation. But the primary difference is that parallax occlusion does not add any additional geometry, nor is it able to affect the silhouette of the mesh. But it's usually much less resource intensive than tessellation. Tessellation is another big one. Red Dead Two has a tessellation quality option, which affects the quality and definition of ground mutations and deformities of things like snow and mud imprints.

News01

Reading the market

15.4s

This is the cliff line, and you can see after we dropped here, we had that magnificent day. The Nasdaq was up 10% or 12%. S P 500 was up 10% that day, but then it stopped, and then it kind of fell off a bit, and then it has retested.

Brand & advertising01

Keeping a community moving

19.5s

Fuel Brindle is a voice and a resource for Iowa's fuel industry. Our members offer a diverse range of products, including fuel, grocery, and convenience items. They help keep Iowans on the move in rural and urban communities. Together, we fuel Brindle.

Signature voices

Four voices. US and UK accents.

04 / ZERO-SHOT VOICE CLONING

Voice cloning

Listen to the reference, then compare RSE-TTS with and without continuation.

Target textReferenceRSE-TTS+ Continuation

The second religion is known as Taoism or the Primitive religion.

Reference
≡

It was later settled as a summer tourist and recreation area.

RSE-TTSSIM 0.782
+ ContinuationSIM 0.779

Back at Stadium mark he once again found himself on the bench.

Reference
≡

The town is adjacent to and borders the city of Amsterdam on three sides.

RSE-TTSSIM 0.772
+ ContinuationSIM 0.811

Secondly, there is a difference between a major tone and a minor tone.

Reference
≡

The butte's trail leads to a panoramic view from the historic fire lookout.

RSE-TTSSIM 0.748
+ ContinuationSIM 0.783

He studied the mammalian chronology of the European Eocene and Upper Miocene.

Reference
≡

Youth Congress always remained in highlights of the leading state news.

RSE-TTSSIM 0.736
+ ContinuationSIM 0.763

Smith is too shocked by Paine's betrayal to defend himself, and runs away.

Reference
≡

The nature reserve covers only a small part of the marsh area.

RSE-TTSSIM 0.728
+ ContinuationSIM 0.754

It is a hermaphroditic fluke that requires two intermediate hosts.

Reference
≡

A live target missile launch was not planned for this flight test.

RSE-TTSSIM 0.720
+ ContinuationSIM 0.708

Through various illegal means, North Texas is awarded an expansion franchise.

Reference
≡

It takes place at fictional Lincoln International Airport near Chicago, Illinois.

RSE-TTSSIM 0.719
+ ContinuationSIM 0.783

Carson greatly admired the talents of Hank Williams.

Reference
≡

The labial palps finally funnel the food into the mouth, where digestion begins.

RSE-TTSSIM 0.717
+ ContinuationSIM 0.666

Institute studies all aspects of autism spectrum disorder.

Reference
≡

Victor Hugo Emerson was the chief recording engineer at Columbia Records.

RSE-TTSSIM 0.714
+ ContinuationSIM 0.696

His name is the same as that of the ancient Egyptian god Amun.

Reference
≡

They are perhaps most known for "The Quill" and the later "Professional Adventure Writer".

RSE-TTSSIM 0.712
+ ContinuationSIM 0.726

More visible enforcement of traffic regulations on motorists, bicyclists, and pedestrians has commenced.

Reference
≡

The resulting wave interference pattern is the basis of diffraction analysis.

RSE-TTSSIM 0.830
+ ContinuationSIM 0.817

An obsessive need to control others is also associated with antisocial personality disorder.

Reference
≡

Systems using artificial neural networks have been used to great effect.

RSE-TTSSIM 0.807
+ ContinuationSIM 0.812

One variation assigns a point based on the pair rolled, rather than the singleton.

Reference
≡

Gum arabic is used primarily in the food industry as a stabilizer.

RSE-TTSSIM 0.801
+ ContinuationSIM 0.811

The meteorite was sent off into the galaxy to be sealed forever.

Reference
≡

His daughter Marcia pursued poetry, while his son Hilary Masters became a novelist.

RSE-TTSSIM 0.801
+ ContinuationSIM 0.747

After the war, May returned to work in the textiles industry.

Reference
≡

The Knights conquered the Prussians and converted them to Christianity.

RSE-TTSSIM 0.797
+ ContinuationSIM 0.842

However, this time it was directly headed by the Russian ambassador.

Reference
≡

York is served by the Toronto Transit Commission's buses and subway system.

RSE-TTSSIM 0.789
+ ContinuationSIM 0.795

His father was a professor of genetic engineering at Louisiana State University.

Reference
≡

The Inca tern feeds primarily on small fish, such as anchovies.

RSE-TTSSIM 0.788
+ ContinuationSIM 0.813

In many cases, such as France, no distinct regional substructures have been employed.

Reference
≡

The CrossLand acquisition gave Washington Mutual a toe hold entry into Oregon via Portland.

RSE-TTSSIM 0.783
+ ContinuationSIM 0.844

The "Review" somehow limped on, buoyed up by a narrow but devoted subscription base.

Reference
≡

Instead, they can change concavity around vertical asymptotes or discontinuities.

RSE-TTSSIM 0.783
+ ContinuationSIM 0.784

The silty bottom of the gulf also contributes to the high turbidity.

Reference
≡

The school's most visible corporate affiliation is with Hillyard, Incorporated a cleaning supplies company.

RSE-TTSSIM 0.782
+ ContinuationSIM 0.784

The King of Portugal was being shaved by his barber.

Reference
≡

Without letters patent, a person is unable to assume an appointed office.

RSE-TTSSIM 0.639
+ ContinuationSIM 0.622

Real schools, secondary schools giving a general practical education.

Reference
≡

In general, people have difficulties estimating risks correctly.

RSE-TTSSIM 0.638
+ ContinuationSIM 0.717

However, a player may purchase as many contestants as they desire.

Reference
≡

They soldiers and officers there are doing God's work, night and day.

RSE-TTSSIM 0.637
+ ContinuationSIM 0.696

Some resemble a tulip or a goblet.

Reference
≡

The school is considered to be both economically and ethnically diverse.

RSE-TTSSIM 0.636
+ ContinuationSIM 0.600

For most owners of the car model, the issue isn't noticeable.

Reference
≡

The population was eighteen thousand four hundred and thirty two.

RSE-TTSSIM 0.635
+ ContinuationSIM 0.699

They can also hike to the panorama restaurant at the top of the mountain.

Reference
≡

The actual amount of oil being shipped would have had to be estimated.

RSE-TTSSIM 0.631
+ ContinuationSIM 0.629

The heather was on fire, and you could see a thin blue smoke rising up slowly.

Reference
≡

Revenge is not my style, but obviously accidents will happen.

RSE-TTSSIM 0.631
+ ContinuationSIM 0.687

Lifeguards were standing on a tower, staring at the water.

Reference
≡

The hotel gives some complimentary water bottles to drink.

RSE-TTSSIM 0.630
+ ContinuationSIM 0.738

In Mexico the term "onza" refers to more than one species.

Reference
≡

The Faculty Board for Teacher Education is also responsible for educational research.

RSE-TTSSIM 0.626
+ ContinuationSIM 0.669

The wooden shrine is generously proportioned for the three images it houses.

Reference
≡

She received an honours degree in English at the University of Cape Town.

RSE-TTSSIM 0.625
+ ContinuationSIM 0.717

To have darshan one has to swim to the temple.

Reference
≡

The eggs are pale yellow when first laid, but become pale brown.

RSE-TTSSIM 0.640
+ ContinuationSIM 0.738

Don't take any chances.

Reference
≡

Many people are traumatized after this sailing accident and need intensive care.

RSE-TTSSIM 0.640
+ ContinuationSIM 0.569

Technically, breaking a mirror is not five years, but seven years of bad luck.

Reference
≡

She wasn't home, a neighbour said that he didn't think she came in last night.

RSE-TTSSIM 0.639
+ ContinuationSIM 0.670

The monastery was inhabited by a succession of three different religious orders.

Reference
≡

His Concerned Citizens ticket won three of the four seats on the city council.

RSE-TTSSIM 0.639
+ ContinuationSIM 0.713

These drawings are usually outline drawings that are quite expressionless and somber in appearance.

Reference
≡

Some ideas discussed during that chat eventually made it into Quake's future public releases.

RSE-TTSSIM 0.639
+ ContinuationSIM 0.716

Singling out alcohol impairment for extra punishment isn't about making the roads safer.

Reference
≡

Frazier considered public education a priority during his first term.

RSE-TTSSIM 0.639
+ ContinuationSIM 0.718

Residents near the river downstream from the slip were evacuated.

Reference
≡

The exclusion of professionals caused several controversies throughout the history of the modern Olympics.

RSE-TTSSIM 0.639
+ ContinuationSIM 0.687

He is the younger brother of actor Jeff Wincott.

Reference
≡

Gandhi feared that the movement might lead to popular violence.

RSE-TTSSIM 0.639
+ ContinuationSIM 0.720

As the video begins, Aguilera is chatting with a group of women.

Reference
≡

This was the first major battle of the Gallic Wars.

RSE-TTSSIM 0.639
+ ContinuationSIM 0.653

It is one of two National Park Service properties in Illinois.

Reference
≡

The expression is now generally used metaphorically.

RSE-TTSSIM 0.639
+ ContinuationSIM 0.733

He went on to lose the general election to Hefner.

Reference
≡

This region is where gold was first discovered in the United States.

RSE-TTSSIM 0.561
+ ContinuationSIM 0.683

Numerous ships were queried and boarded to verify their cargo manifests.

Reference
≡

Each character has a role such as medic, sniper, scout, grenadier, soldier or engineer.

RSE-TTSSIM 0.559
+ ContinuationSIM 0.562

Without understanding the actual physics, some animals exploit total internal reflection for catching fish.

Reference
≡

The rainforests are depleted, we turned the planet into a waste ground.

RSE-TTSSIM 0.559
+ ContinuationSIM 0.618

The people listed below have featured on Gabonese stamps since independence.

Reference
≡

Spring described the offer as "irresistible" and the appointment proved very successful.

RSE-TTSSIM 0.557
+ ContinuationSIM 0.701

It is situated to the north of the centre of Darlington.

Reference
≡

Some of the popular instruments used to be guitars, violins, accordions, and marimbas.

RSE-TTSSIM 0.555
+ ContinuationSIM 0.610

The only shadow was that of the few scattered pine trees.

Reference
≡

He approached the mass and was surprised at the size and the shape.

RSE-TTSSIM 0.553
+ ContinuationSIM 0.649

He then visited Italy for a few months to regain his health.

Reference
≡

The strain of playing almost constantly took a physical toll on Casals.

RSE-TTSSIM 0.553
+ ContinuationSIM 0.709

The irrigation system will be fixed within a week.

Reference
≡

The first concern is to patch the holes in the ship, then we can pump out the water.

RSE-TTSSIM 0.548
+ ContinuationSIM 0.691

She can also hide her young in the leaves while searching for food.

Reference
≡

He is also a Patron of Creativity Australia.

RSE-TTSSIM 0.547
+ ContinuationSIM 0.608

For the full history, see List of Southern Conference men's basketball champions.

Reference
≡

The engine does not use titanium valves or connecting rods.

RSE-TTSSIM 0.542
+ ContinuationSIM 0.466

A couple walks through an Asian restaurant.

Reference
≡

This a picture of a man cleaning off a diving board with a broom.

RSE-TTSSIM 0.561
+ ContinuationSIM 0.660

They defend themselves if they feel threatened, but otherwise tend to ignore humans.

Reference
≡

He is patron of the Shandar Smith Foundation, a charity for children with cancer.

RSE-TTSSIM 0.560
+ ContinuationSIM 0.739

A few languages such as Haskell implement them as linked lists instead.

Reference
≡

Previously an attorney, Korona left the profession to pursue a career in music.

RSE-TTSSIM 0.560
+ ContinuationSIM 0.589

This principle is a restriction but also is a guide in formulating physical theories.

Reference
≡

Eleven nations participated in the experiments.

RSE-TTSSIM 0.560
+ ContinuationSIM 0.684

It is believed to feed on insect larvae and termites.

Reference
≡

Additional support is also available to agencies through the Center's Forensic Services Unit.

RSE-TTSSIM 0.559
+ ContinuationSIM 0.630

To summarize, our comprehensive survey discloses the true versatility of their method.

Reference
≡

Before we begin with the actual baptism, let's renew our creed.

RSE-TTSSIM 0.558
+ ContinuationSIM 0.615

The first office was in the basement of the Presbyterian Church’s university campus ministry.

Reference
≡

Engine and automatic transmission are located in the left rear corner of the bus.

RSE-TTSSIM 0.558
+ ContinuationSIM 0.647

Strike while the iron is hot Make hay while the sun shines

Reference
≡

We pride ourselves in completing all requests in less than twenty four hours.

RSE-TTSSIM 0.557
+ ContinuationSIM 0.597

This means that generic terms are generally not afforded any legal protection.

Reference
≡

He was seen as an idealistic, but also polarizing politician.

RSE-TTSSIM 0.556
+ ContinuationSIM 0.643

This artful creation is more than just a picture, it's a statement by the artist.

Reference
≡

The autonomous ship floated closer to receiving its flying cargo.

RSE-TTSSIM 0.556
+ ContinuationSIM 0.687

His wife Ginny survives him with their son, two grandchildren and a great grandchild.

Reference
≡

The Knights conquered the Prussians and converted them to Christianity.

RSE-TTSSIM 0.794
+ ContinuationSIM 0.823

05 / PAPER

Read the research.

Explore the method and results.

Download PDF ↗
BibTeX
@misc{ma2026rsetts,
  title = {RSE-TTS: Compact Text-to-Speech through Shared Text-Aware Reference Encoding and Generation},
  author = {Ma, Ming and Chen, Mingjie and Zheng, Yuanzhong and Bao, Hengyao and Yu, Hailin and Yu, Shijing and Qiu, Hao and Cao, Gongfan and Li, Xiao and Zhao, Xuemin and Jiang, Youhai},
  year = {2026},
  note = {Research manuscript}
}