Learn how Laravel chunk upload helps you efficiently handle large file uploads by splitting them into smaller, manageable chunks. This method reduces the risk of timeouts and network issues commonly associated with uploading large files in one go. Once all chunks are successfully uploaded, they are combined into a single file and securely stored in an S3 bucket, ensuring smooth and reliable file management.
Step-by-Step Guide to Implement Laravel Chunk Upload
Follow this step-by-step guide to integrate Laravel chunk upload into your app. We’ll walk you through the process of setting up Laravel chunk upload configurations, uploading large files in chunks, and ensuring secure file handling
Key Purposes:
1. Receives and Stores File Chunks
– Each chunk is stored temporarily in storage/app/temp/chunk/{uuid}/{chunk_id}.
2. Ensures the Upload Progress is Tracked
– Uses a cache system to store the file name and track the upload progress.
3. Assembles the Chunks Once All Are Uploaded
– Calls areAllChunksUploaded($uuid, $chunk_count) to check if all parts are uploaded.
4. Triggers a Background Process for Final Upload
– Runs an Artisan command (chunks:uploads3) asynchronously to merge the chunks and move the final file to an S3 storage.
vue2-dropzone(for front-end)
For the frontend, we are using Vue.js, and to simplify the file upload process, we are leveraging vue2-dropzone.
This is a Vue.js wrapper around the popular Dropzone.js library, making it easy to implement a drag-and-drop file upload feature in Vue 2 applications. It provides built-in support for chunked uploads, progress tracking, and error handling, ensuring a smooth user experience when uploading large files.
Key Features:
✅ Drag-and-drop or click-to-upload files
✅ Supports multiple files
✅ Shows a preview of uploaded files
✅ Allows customization (like accepted file types, max file size, etc.)
Create a Vue component(add_video.vue):
<template>
<div class="col-md-6">
<div class="form-group">
<label for>Add Video</label>
<div>
<vue-dropzone id="video-upload"
v-on:vdropzone-upload-progress="updateProgressDz"
v-on:vdropzone-file-added="fileAddedDz"
v-on:vdropzone-success="afterFileSentDz"
:options="dropzoneOptions"
></vue-dropzone>
</div>
<div id="uploadSpeed">
<i></i>
</div>
<div class="upload-progress-slider p-0">
<div class="progress simple">
<div
class="progress-bar"
id="video-progress"
role="progressbar"
:aria-valuenow="false"
aria-valuemin="0"
aria-valuemax="100"
></div>
</div><div id="percentage"></div>
</div>
</div>
</div>
</template>
<script type="text/babel">
import vue2Dropzone from 'vue2-dropzone'
import 'vue2-dropzone/dist/vue2Dropzone.min.css'
export default {
name: "add_video",
components : {
vueDropzone: vue2Dropzone
},
data() {
return {
progress: 0,
formData: null,
busy: false,
initUploadTime: null,
dzFileSize: 0,
dropzoneOptions: {
addRemoveLinks: true,
canceled: function(){
window.location.reload();
},
dictDefaultMessage: "*.mp4, .pdf files only",
url:`/video/dz_video_upload`,
method: "post",
acceptedFiles: ".mp4,.pdf",
maxFiles: 1,
timeout: 180000,
maxFilesize: 200,
chunking: true,
forceChunking: true,
chunkSize: 2048000,
retryChunks: true,
retryChunksLimit: 3,
parallelChunkUploads: false,
}
};
},
props: {
video_id: {
type: [String, Number],
required: false,
default: null
},
uploading: {
validator: val => ["video"].indexOf(val) !== -1
}
},
mounted() {
this.formData = new FormData();
},
beforeMount() {
window.addEventListener("beforeunload", event => {
if (!this.busy) return;
event.preventDefault();
// Chrome requires returnValue to be set.
event.returnValue = "";
});
},
methods: {
fileAddedDz(file){
this.dzFileSize = file.upload.total / 1000;
this.initUploadTime = (new Date()).getTime();
},
afterFileSentDz(file, response) {
response = JSON.parse(file.xhr.response);
$("#uploadSpeed i").hide();
$("#percentage").html('');
$("#video-progress").css({display: 'none'});
const data = response.data;
this.busy = false;
if (data && data.video) {
if (data.video.extension == 'pdf') {
this.$toaster.success("File Successfully Uploaded!");
} else {
this.$toaster.success("Video Successfully Uploaded!");
}
} else {
this.$toaster.error("Unable to upload video file");
clearInterval(checkProcessed);
}
},
updateProgressDz(file, progress, bytesSent) {
let kbSent = this.dzFileSize * progress / 100;
if (progress === 100 && bytesSent / 1000 !== this.dzFileSize) {
return;
}
let progressBar = $("#uploadSpeed i");
let now = (new Date()).getTime();
let time_passed = (now - this.initUploadTime) / 1000;
let kbps = kbSent / time_passed;
if (kbps < 1) {
progressBar.html(`${(kbps * 1000).toFixed(2)} B/s`);
} else if (kbps < 1024) {
progressBar.html(`${(kbps).toFixed(2)} KB/s`);
} else {
progressBar.html(`${(kbps / 1024).toFixed(2)} MB/s`);
}
$("#percentage").html(`${progress.toFixed(0)}%`);
$("#video-progress").css({width: `${progress}%`});
},
}
};
</script>
Note: Please install the toaster library before using this code. If you are using a different notification library, replace this.$toaster.success("File Successfully Uploaded!"); with the corresponding method from your library to avoid errors.
Inside theweb.php Create route:
Route::post('video/dz_video_upload', 'VideoController@videoChunkUpload');
Create videoChunkUpload function inside the App\Http\Controllers\VideoController
public function videoChunkUpload(Request $request)
{
$file = $request->file('file');
$uuid = $request->input('dzuuid'); // Unique identifier for the upload session
$chunk_id = $request->input('dzchunkindex'); // Current chunk index
$chunk_count = $request->input('dztotalchunkcount'); // Total number of chunks
$extension = $file->getClientOriginalExtension(); // File extension
// Check if the filename is already cached
if (Cache::has($uuid)) {
$filename = Cache::get($uuid);
// Refresh cache timeout (2 minutes per chunk)
Cache::put($uuid, $filename, 2);
$original_name = 'my-video.mp4';
if ($file->getClientOriginalName()) {
$original_name = $file->getClientOriginalName();
}
} else {
$original_name = preg_replace("/[^a-z0-9\_\-\.]/i", '', basename($file->getClientOriginalName()));
// Generate a unique filename with timestamp
$filename = sprintf('%s-%s-%s', time(), $original_name);
Cache::put($uuid, $filename, 2);
}
// Store chunk file in local storage
if ($request->hasFile('file')) {
Storage::disk('local')->put("temp/chunk/$uuid/$chunk_id", $file->get());
} else {
throw new \Exception('Chunk failed.');
}
$video = [
'chunk_count' => $chunk_count,
'filename' => $filename,
'uuid' => $uuid,
'original_name' => $original_name,
'extension' => $extension,
];
// Check if all chunks are uploaded
if (self::areAllChunksUploaded($uuid, $chunk_count)) {
// Execute the command asynchronously to merge chunks and upload the final file
popen("php " . addslashes(base_path('artisan')) . " chunks:uploads3 $chunk_count $filename $uuid videos > /dev/null 2>&1 &", 'r');
return ['message' => 'Video uploaded successfully', 'video' => $video];
} else {
return ['message' => 'Chunk Uploaded', 'video' => $video];
}
}
private static function areAllChunksUploaded($uuid, $chunk_count)
{
// Get all uploaded chunk files from storage
$files = Storage::disk('local')->files("temp/chunk/$uuid");
// Loop through all expected chunk indexes and verify if each chunk is present
for ($i = 0; $i < $chunk_count; $i++) {
if (!in_array("temp/chunk/$uuid/$i", $files)) {
return false;
}
}
return true;
}
Suggestions Ensure Cache::put($uuid, $filename, 2); has a Proper TTL(Time To Live)
In my example I’m setting a cache timeout of 2 minutes per chunk, which might not be enough for large file uploads. If you are using this logic for extra large files then replace like this:
Before
Cache::put($uuid, $filename, 2);
After
Cache::put($uuid, $filename, now()->addMinutes(10))
To merge all chunks and upload the final file to S3 storage, create a new file at app/Console/Commands/JoinChunksAndUpload.php.
<?php
namespace App\Console\Commands;
use Illuminate\Console\Command;
use Illuminate\Support\Facades\Storage;
class JoinChunksAndUpload extends Command// Define the temporary file path where the merged file will be stored
{
/**
* The name and signature of the console command.
*
* @var string
*/
protected $signature = 'chunks:upload {total_chunks} {file_name} {uuid} {type}';
/**
* The console command description.
*
* @var string
*/
protected $description = 'Join Chunks And Upload To S3';
/**
* Create a new command instance.
*
* @return void
*/
public function __construct()
{
parent::__construct();
}
/**
* Execute the console command.
*
* @return mixed
*/
public function handle()
{
$total_chunks = $this->argument('total_chunks');
$file_name = $this->argument('file_name');
$uuid = $this->argument('uuid');
$type = $this->argument('type');
// Define the temporary file path where the merged file will be stored
$file_path = "user-uploads/temp/$file_name";
try {
// Ensure an empty file is created before appending chunks
Storage::disk('local')->put("temp/$file_name", '');
// Open the file in append mode to start merging chunks
$file = fopen(base_path("public/$file_path"), 'a+') ;
// Loop through all chunks and append them to the main file
for ($i = 0; $i < $total_chunks; $i++) {
$content = file_get_contents(base_path("public/uploads-file/temp/chunk/$uuid/$i"));
fwrite($file, $content);
}
fclose($file);
// Upload the merged file to S3 storage
Storage::disk('s3')->put("$type/$file_name", \fopen(base_path("public/$file_path"), 'r+'));
// Delete the temporary chunk directory after successful upload
Storage::disk('local')->deleteDirectory("temp/chunk/$uuid");
// Remove the merged temporary file from local storage
unlink(base_path("public/$file_path"));
return ['message' => 'File uploaded successfully', 'video' => $type/$file_name];
} catch (\Exception $exception) {
// If an error occurs, delete the chunk directory to clean up storage
Storage::disk('local')->deleteDirectory("temp/chunk/$uuid");
throw new \Exception($exception->getMessage());
}
}
}
In this code, all chunks are merged and stored both the video inside the S3 bucket and local storage.
If processing takes time and you don’t want to show a loader on the frontend, you can create a job to handle it in the backend. For example, when a user uploads a video, you can display a success message using a toaster notification while the backend processes the video.
However, if the video size is moderate—not too small or too large—you may choose to show a loader on the frontend during the upload process. This approach ensures a better user experience.
For more insightful tutorials, visit our Tech Blog and explore the latest in Laravel, AI, and Vue.js development!
