Talking AI Clones · ElevenLabs Voice + HeyGen Avatar Workflow

Quick Summary

• Jonathan Acuña's recommended stack separates the two hardest parts of a talking clone: ElevenLabs generates the voice, while HeyGen turns a script or audio track into a performance by a Digital Twin avatar.
• For fast tests, ElevenLabs Instant Voice Cloning can work from short samples. For a production identity, Professional Voice Cloning is the more relevant path. ElevenLabs recommends at least 30 minutes of clean, single-speaker audio, at least one hour for stronger results, and ideally closer to two or three hours.
• HeyGen currently recommends a continuous 2 to 5 minute avatar recording at a minimum of 1080p and 30 fps, with even lighting, clear synchronized speech, direct eye contact and controlled movement. A separate consent recording from the same person is required.
• The creator prefers Multilingual v2 over v3 for this use case. Current ElevenLabs documentation now lists v4 and v4 Turbo alongside v3, Multilingual v2 and Flash v2.5. Treat model selection as a test against your own voice, language, script length and delivery rather than a permanent rule.
• HeyGen can connect directly to an ElevenLabs voice, or accept a custom audio file. Generate and approve the voice first, then let the voice track become the timing reference for the avatar performance.
• The strongest use of an avatar is continuity, not deception or fully autonomous publishing. Keep human approval over scripts, pronunciations, claims, final renders and publication, and make synthetic use clear whenever the context could mislead viewers.

The useful idea: split the clone into voice and performance

A talking clone is not one model. It is a chain. The voice system must reproduce timbre, cadence, accent and delivery. The avatar system must reproduce face, movement, eye contact and lip synchronization. Acuña's recommendation assigns those jobs to ElevenLabs and HeyGen instead of asking one platform to be equally strong at both.

This separation also improves revision. A weak sentence can be regenerated in ElevenLabs without retraining the avatar. A visual problem can be fixed in HeyGen without rebuilding the voice. The approved audio becomes the stable bridge between the two systems.

Credit: the embedded tutorial and its examples belong to Jonathan Acuña, Doctor AI. The implementation details and product checks in this article use current ElevenLabs and HeyGen documentation available through October 21, 2026.

Choose the right ElevenLabs cloning method

ElevenLabs offers Instant Voice Cloning and Professional Voice Cloning. Instant cloning uses a short reference at generation time and can produce a useful result quickly. It is appropriate for prototypes, private experiments and situations where only a few minutes of clean audio are available.

Professional Voice Cloning fine-tunes a dedicated model on a much larger dataset. It requires a Creator plan or above and a voice-verification step. It is the stronger choice when the clone represents a person or brand repeatedly, especially across different scripts and delivery styles.

Acuña recommends at least one hour of recorded speech. That aligns with the official workflow, which asks for at least 30 minutes, recommends at least an hour in the upload interface, and says results can improve as the dataset approaches two or three hours. More audio is useful only when it remains clean and stylistically consistent.

Record voice training material that the model can actually learn

Use one speaker, one microphone position and one intended performance style. Record in a quiet, dampened room without music, overlapping voices, aggressive noise reduction or changing room acoustics. A pop filter, stable gain and a consistent distance from the microphone reduce defects that the clone might otherwise learn.

Acuña calls an external microphone mandatory. ElevenLabs is less absolute: its documentation says any microphone can work, while recommending a good XLR microphone and audio interface for optimal results. The practical standard is not the price of the microphone. It is clean, undistorted, consistent audio with minimal room echo.

Read material that resembles the content you plan to publish. A calm educational clone needs calm educational training material. Include the names, vocabulary, abbreviations and language patterns your scripts will use. Do not combine a quiet narration style, a loud sales voice and casual conversation into one dataset unless the selected cloning workflow explicitly benefits from that range.

Do not turn one model preference into a permanent rule

In the video, Acuña says he prefers version 2 and dislikes version 3 for his cloned voice. That is a creator-tested preference, not a universal benchmark. Earlier ElevenLabs guidance also noted that Professional Voice Clones were not fully optimized for the v3 research preview, which helps explain why some users preferred Multilingual v2 for stability.

The current model list has moved on. Eleven v4 and v4 Turbo are now the flagship expressive models, v3 remains available for dramatic performance, Multilingual v2 is positioned as stable for long-form generation, and Flash v2.5 prioritizes speed and cost. Test the same short script across the models available to your account, then score identity, pronunciation, pacing, emotional control and consistency.

Keep a small benchmark script that contains a normal sentence, a difficult name, a number, an acronym, a question and an emotional shift. Re-run it when a model changes. A model that wins on one promotional line may lose on a three-minute educational video.

Capture a strong HeyGen Digital Twin

HeyGen's current Digital Twin guidance recommends at least two minutes and up to five minutes of continuous footage. Record at a minimum of 1080p and 30 fps, use bright even lighting, keep the background static, look into the lens and make sure only the subject's voice is audible.

The source performance teaches the avatar how to move. Controlled gestures, natural expressions and brief closed-mouth pauses give the system useful behavior to reproduce. Sudden head turns, hands crossing the face, repeated glances away from camera and busy clothing can create artifacts or teach an awkward performance pattern.

Do not remove the source audio even when ElevenLabs will supply the final voice. HeyGen states that speech in the avatar footage is important for learning lip synchronization. The final ElevenLabs track can be connected or uploaded later.

Consent is part of the production pipeline

HeyGen requires a short consent video for every video-based Digital Twin. The consent must come from the same person shown in the avatar footage, use the displayed script and be recorded with a visible face and clear synchronized audio. If you are creating an avatar for someone else, that person must record their own consent.

ElevenLabs also verifies Professional Voice Clones. Its documentation says the platform currently allows users to clone their own voice and uses voice verification before training. Technical verification does not replace contracts, client approval or local law. Record what the person approved, where the clone may appear, how long permission lasts and how access will be revoked.

Do not create a client clone by opening an account on their behalf and treating access as permission. Define account ownership, credential custody, voice and avatar rights, allowed scripts, deletion, incident response and who must approve each publication before production begins.

Connect ElevenLabs to HeyGen

HeyGen documents a direct third-party voice integration. In AI Studio, select an avatar, open the voice controls, add a new voice, choose the third-party integration and connect ElevenLabs. This makes eligible voices available inside the HeyGen project.

The alternative is an audio-first handoff. Generate the approved narration in ElevenLabs, export it, then upload the custom audio to HeyGen. HeyGen accepts custom audio for avatar dialogue in its studio tools. This approach is easier to audit because the exact approved audio file can be archived with the script and final render.

Never expose an API key in a shared prompt, project file, public repository or client handoff. Use the platform's integration screen or a secure secret store. Give each client or project the smallest necessary access and rotate keys when a collaboration ends.

A repeatable talking-clone workflow

Step 1: define the audience, channel, target duration and one useful outcome. Step 2: draft the script and verify every claim. Step 3: generate a voice preview and correct pronunciation, pauses and delivery. Step 4: approve and lock the audio. Step 5: place that audio in the HeyGen scene with the correct Digital Twin and look.

Step 6: generate one low-risk test. Step 7: review lip sync, facial stability, eyes, teeth, hands, clothing, background, captions and the transition between generated and real assets. Step 8: add B-roll only when it explains or proves something. Step 9: obtain human approval. Step 10: publish through the normal channel workflow and retain the approved script, audio and render together.

If an error appears, fix the earliest responsible stage. A pronunciation problem belongs in the voice file. A pacing problem belongs in the script or audio. A visual artifact belongs in the avatar take or look. Rebuilding the entire pipeline for one local defect wastes time and makes regressions harder to detect.

The clone is not the content strategy

The most valuable part of Acuña's argument is the search for a repeatable format. Before scaling, identify a stable hook, camera distance, title treatment, caption language, duration, value delivery and call to action. The avatar then reproduces a format that already communicates well.

Do not confuse one viral example with proof that the format will work for every account. Compare avatar posts with human-recorded posts using the same topic family and distribution conditions. Track completion rate, saves, qualified replies, profile visits and conversions separately from views.

Acuña frames avatar content as coverage for days when the creator cannot record. That is a stronger starting point than replacing all human appearances. A hybrid feed preserves real performances, live reactions and trust while using the avatar for consistent explainers, updates, localized variants and approved evergreen material.

Consistency matters, but volume is not a quality metric

The video advocates a very high posting quota and cites a broad expectation that exposure will create sales. Those statements are motivational opinions, not verified conversion guarantees. Publishing more can create more opportunities to learn, but it can also multiply weak ideas, review work and platform fatigue.

Start with a cadence the team can review properly. Batch three to five scripts, render them, compare the finished pieces and publish only the versions that meet the rubric. Increase volume after the process produces reliable quality and measurable audience value.

Automation should stop before publication unless the brand has already approved the exact content and operating rules. Keep private messages, claims, sponsorship disclosures, regulated topics and sensitive replies under human control.

Use a fixed quality rubric before every post

Score voice identity, pronunciation, pacing, lip sync, facial stability, gesture quality, visual continuity, caption accuracy, factual accuracy and disclosure from 1 to 5. A failure in consent, factual accuracy or identity should block publication regardless of the average score.

Review once with sound, once without sound and once at mobile size. The silent pass reveals caption and visual problems. The audio-only pass reveals pacing, pronunciation and unnatural breaths. The mobile pass exposes small type, weak eye contact and framing that looked acceptable on a desktop monitor.

Keep the failed versions. They become a useful benchmark library showing which microphone setup, script style, model and avatar look caused each defect. Production improves faster when the team can compare evidence instead of relying on memory.

Starter checklist

Before recording: confirm ownership and consent, choose Instant or Professional Voice Cloning, decide the target language and delivery style, prepare clean recording space, and write training material that matches the intended content.

Before generating: approve the script, verify claims, test difficult words, select the voice model, choose the avatar and look, confirm aspect ratio, and decide whether the post needs an AI disclosure.

Before publishing: compare the render with the approved audio, run the quality rubric, verify captions and links, obtain human approval, and store the final script, audio, render and consent record according to the agreed retention policy.