DTube Uploader, born to be alive!

in dtube •  6 months ago

Before I start talking about the uploader, I have to talk to you about how I joined the DTube team.
It goes back to 6 years ago! (Yes I know DTube to less than 1 year ^^).
It started with the meeting of @Heimindanger as an employee both in a company where have developed an eCommerce website.
We worked together a good year before he flies his own wings!
We kept a good distant relationship.
One day he contacted me and asked me if I wanted to participate in the project, he explains the project, I did not know Steem, IPFS, Meteor, crypto-currencies... I thought about it and I said yes!

The different versions

I started contributing to DTube on version 0.3 4 months ago with the possibility of translating DTube into several languages https://steemit.com/dtube/@heimindanger/dtube-0-3-this-update-contains-so-many-new-features-i-could-not-find-a-good-title-for-it.

Then, I hooked up with the creation of any part of the uploader for version 0.5 only 2 months ago https://steemit.com/dtube/@heimindanger/d-tube-0-5-big-optimizations.

Finally the possibility to have a preview of the video on the scrollbar, as well as the ability to send the Snap to resize and embed the new logo DTube, there is 1 month in v0.6 https://steemit.com/dtube/@heimindanger/d-tube-0-6-pushing-it-to-the-limit.

My first article!

If you followed, I'm quite new to Steem and it's also my first article on the internet, thanks in advance for your indulgence ^^

The subject of this article is about the uploader! I decided to do this article because many people are wondering about the operation, I thought it was a good reason to dive into the bath for this first article!

Why uploader?

To answer problems that users were facing and because it was not possible easily in decentralized mode!

The first need was to encode the source videos in a smaller format (480p) allowing more people to play videos with a low internet speed.

The 2nd was to have the possibility to have a progress bar when sending the video file.

We then took advantage (having on hand the source video!) to encode the video in a video and audio format allowing to be read by the largest number of browsers on the market (H264 for video and AAC for audio).
And this allowed us to take the opportunity to directly send the source video and 480p video directly into IPFS and thus lighten the customer of this long and unstable task at the time.
In version 0.5, we then added the creation of a video Sprite allowing to visualize thanks to the scroll bar of the video the preview of the moment of the video.

Let’s go!

Here is the block-by-block explanation of the operation of the uploader that allows you to send videos to DTube!

Upload video

b/ Sending the video to the server with progress bar
f/ Token recovery for encoding tracking, IPFS hash recovery, ...

The heart of the Uploader

For fast processing after the end of the upload, everything is treated as asynchronously and in parallel, there is only one token that is returned with prior requests for creation of Sprite (d), encoding (e) performed …

Encoding Daemon

Daemon encoding can handle 2 different tasks, creating Sprites files and 480p encoding of the source file.

How the Sprite is created to preview the video:

  • Extraction of 100 images of the source video in 210 * 118 respecting the ratio
  • Combining 100 images vertically => allows to see the first images if the image is slowly loading!
    Note: if the video is less than 100 seconds, the extraction of images is done with 1 image per second, which is more than enough.

For 480p encoding operation, the source video is simply converted to 854 * 480 for 16: 9 video.
Video conversion to H264 is also performed as well as audio conversion to AAC. Thus this 480p video can be read by many people and by several browsers, unlike the source video.

Once each file is ready (Sprite or 480p file), it is placed in the IPFS queue.
The progress of each treatment is updated every second so that the user can follow it.

IPFS Daemon

The file is added to the decentralized IPFS network.
Once the user recovers all the hashes via repeated calls, the video article can be posted on the Steem network.

Finally on the home page

The video list is extracted from Steem's blockchain, and each video contains an url link from IPFS, hosting the Snap, video, Sprite...

Global scheme !

Extra: the Snap and the DTube logo inlay

Steps:

  • Simultaneous scaling and cropping for a final image of 210 * 118
  • Adding in IPFS this final image
  • Adding IPFS to this image with the DTube logo overlay

Optimizations

If a user had to leave the upload page after sending the video, 480p video encoding and adding to IPFS would be useless and valuable resources would be lost.
That's why if the upload page is closed, the tasks associated with this source video on the video encoding, the creation of the Sprite, the addition in IPFS ... are canceled!
This point saved a lot of resources several weeks ago.

Limitations

The 480p encoding and the creation of the Sprite is for the moment limited to videos less than 20 minutes for cost reasons. Which is still the majority of videos sent on DTube.
The encoding of the source video is currently limited to 480p for cost reasons as well.
Note: These are things that will evolve and change in the future!

The source video

The source is kept as is, without any transformation and limitation in format, size, duration ... but we must pay attention to browsers and Internet speeds of users who may not be able to view the video and not vote at once!
A video with a smaller size, and an encoding in h264 for video and AAC for audio will allow all to see your source video, think about it!

Shortly

The uploader is evolving!
Since the last version v0.6 things have accelerated, the affluence helping and the technical challenges that grow with, and it's rather motivating!
The next article will explain these changes.

Conclusion

This is my very first article that I publish on the internet and also on the blockchain Steem, be indulgent!
Thanks to @heimindanger for his tips for writing articles!
Feel free to leave your comments. I’m french, so thanks to google translate!

Links

DTube: https://d.tube
Uploader: https://github.com/dtube/ipfs-uploader

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

I know how hard you guys go through with all these. I’ve been using Dtube since I learned about it there still a lot to improve here but i’ll be glad to see dtube be in a different level soon. Thanks for all your hard work to make dtube progress and ensure quality.

·
·
·

Why can I not search d.tube or post vids from my iPhone?

Dtube is growing on the inside let's feed it with the original content

Thanks for sharing the inside on the improving of the Dtube's uploader. Tried it out today and had a very positive smooth experience. I believe that as much as Dtube needs to constantly upgrade its tech it should also be looking towards raising the quality standards of its platform, potentially looking into creating its own original content.

And I am extremely happy to start contributing content to my Dtube channel. I'm currently working on a documentary about the Crypto Utopia being built in Puerto Rico currently and am wondering would you guys (@dtube, @heimingdange, @superkoala and others) would like to discuss the potential of launching it as a Dtube original project.

This episode is a pilot for the web-documentary about the way Crypto changes our world. Shot in the lean format this project is able to catch the trends faster than the other traditional big media sources documenting the history in making.

Merci,
-K.

It's pretty amazing to see the behind the scenes of the video processing that DTube and the uploader do.

People don't realize how complicated this stuff really is, especially when dealing Peer To Peer. Thanks for all your hard work and I can't wait until we see even more improvements and quality adjustments!

·

Yes and it's a team work!

Loading...

Welcome , you did pretty good for your first article and also thank you. If dtube is going to take over its going to need all the brain power it can get, much success.

·

Thank you

Just a thought. Maybe a Windows and Linux self installer and auto updater for those of us who wish to contribute processing power? Say I want to donate 20% or so of one of my processing cores and 10% of my GPU to process videos.

Additionally an option for redundancy for video files itself, not so much like Torrents but very similar like how Freenet Project handles websites/FreeSites for sake of example?

I'd willingly donate some processing power from my gaming rig and roughly 50 gigs of HDD.

·

Another approach is in progress to contribute.. a desktop version in windows linux and ios to make what the uploader do but on your computer!

·

You can also run a ipfs node to contribute for the ipfs network if you want

·
·

I'm aware I could contribute to the IPFS network voluntarily, what I'm getting at though is that there are likely people who would contribute in some size, shape or form, but when it comes to installing and setting up IPFS they get deer in headlights syndrome. Even some of the more technically savvy end up that way ;)

So all the same, having an installer that would essentially do the basic installation would be beneficial. Or does one exist now since the last time I checked and I'm just being an idiot? =\

·
·

Huh https://dist.ipfs.io/#go-ipfs

Wouldn't you know it =\ things have progressed a hell of a lot since the last time I peeked at this stuff.

·
·

Is there a tutorial on how to do this? Also if you host your own stuff on your own IPFS node can you achieve greater than 480p? Thanks for your awesome work!

·
·
·

Yes there is an tutorial, i found one for german: https://d.tube/#!/v/sempervideo/7rsqfn1d

You really landed with the right foot into the Blockchain. Great improvements DTube is having... as always, everything is a matter of time!

Cool to have you in here @superkoala

·

Thank you!

Thank you for your AMAZING work on Dtube it's a HUGE part of this community!

Welcome to steem @superkoala! :)

And thank you for your contribution to dtube!

·

Thank you!

@superkoala Man, you guys are amazing!!! I have seen the progress from the beginning, and it is GREAT. Looking forward to big things, and I am recruiting to DTube every chance I get. :-)

If I may ask I'm very curious as to the availability of infrastructure that DTube has? As you may know, YouTube has 400 hours of video uploaded every minute. Granted, I don't anticipate you guys dealing with that load factor anytime soon, but I'm curious how you guys are set up.

I'm incredibly excited about the future of DTube, however I'm keeping a cautious outlook as I've seen platforms try and compete with YouTube, although I'm not suggesting that's what you guys are trying to do, I've seen platforms have to close shop because they just didn't have the capacity to handle all the traffic.

Thanks for providing this environment for us. My experience here has been phenomenal and I'm very excited about contributing and participating.

·

Thank you! You are right and we know that very very well 😀 it is our first priority since the start and we work hard on it. If you see now it is ok. We increase hardware when we reach a limit for cost reason and we have a lot of idea to manage more and more trafic. We have idea to increase trafic too but we delay it to keep the platform working 😉

When will dtube support videos on 720 or 1080 ?

Good decision to work together with him.

@superkoala you were flagged by a worthless gang of trolls, so, I gave you an upvote to counteract it! Enjoy!!

thanks for your information

Wow, thanks for the information! Always you give good information that's why i follow you!🔝🔝🔝🔝

·

Thank you!

I know this is slightly off topic but still related to Dtube. Does anyone else have a real difficulty trying to get videos to play smoothly? Every time I try to watch a video it either freezes after a few seconds or starts doing a really jumpy buffering where audio comes in and out and the video plays for like 1 second and then stops for a second and this just goes on and on. I found an app on PlayStore today for Dtube so I thought I would try that to see if this was any better but although the issues were slightly different it was still pretty much unwatchable. It's a shame as this app had support from Chromecast too. Before anyone asks about bandwidth I have 80Mb down and 50Mb Up so this isn't the issue and YouTube vids play just fine. Hope this is something that can be fixed as I really wanna watch some of these videos.

·

It is very strange with your bandwith, where do you come from ?

Really cool job!
Nice to show that big part of your life to us

Thank you for your work on Dtube! Great article... for an internet publishing newbie... Ha ha!

·

Yes it is really my first article. But @heimindanger help me with tips and advices, thanks again for him

Very well explained! And great graphics.

·

Thank you! Powerpoint power ahah!

Hyy
Guys this is technical Asia
I am new to dtube I want your support👍💯🙏

Glad to have you on the team! I'm really looking forward to being able to use DTube, because I've been unsuccessful the past fifteen times I've tried to upload... Error submitting to blockchain. I am able to upload video, then upload the snap, but when I enter the text description, etc, and click submit, i get the blockchain error... Until then I will continue using Youtube, since I do 4K UHD drone videos. Cheers.

·

I’ve written about this issue and how to get through to upload your video to @dtube
Check this out @texagonia
https://steemit.com/dtube/@blascobite/dtube-upload-error-and-how-to-get-through

·

hummm... I think it is not uploader issue ... but steem, try report here : https://discordapp.com/channels/347020217966395393/400281011826458635

Great job and welcome to steemit! :)
Maybe you can help me out, I've never been able to upload any videos edited using Windows Movie Maker. Is this because WMV files are not supported?

thx for this info...

Nice work

This is amazing! your post is the best many Dtubers can learn about this like me. Thank you for the contribution Dtube

·

Thank you

Rather informative and I was honstly curious how that actually worked. Thanks for the blow by blow on the uploader and how videos were done. Informational itch has been scratched.

·

Yes I try to keep it easy to read and to understand because it is a little bit complicated^^

·
·

Far more complicated than the stuff I've worked on in the past, that much I'll say. Damn dude I'm a physicist and this kind of process warps my brain a little bit trying to think deeper into the nitty gritty. I commend you on your hard work, dedication and devotion to the projects.

·
·
·

Thank you!

Using C# for the encoding daemon I see … "Interesting" choice.

I'm wondering how many servers you're running right now for the encoding service (and where. Azure? AWS?). How well does it scale, etc.

·

Yes C# forever :-) Now, we have 3 to 4 server for encoding with a third party which accept steem ^^

Hello, I need some help with d.tube. Sor far, I can use d.tube for watching rating videos and comments. But when it comes to write or replay to a comment below a video on d.tube, it doesn't work for me. When I click "add a comment" or "reply to..." it doesn't do anything.

I use Chrome with a script blocker so I thought first maybe it blocks something, but still, after I switched it off, it didn't work. Another person recommended me to use steemconnect instead of my private posting key but still no success. So far I've been writing comments directly on steemit.com but I would prefer to use the d.tube interface. Do you have some other ideas or tips for me?

·

It is a front web side issue, perhaps @heimindanger or @hightouch can help you

·
·

I found the source for the problem. After deactiviating the extension "Grammarly for Chrome" it works now. Obviously d.tube uses a different text area than steemit.com.

·
·
·

Thank you for sharing

·
·

would you be able to help me too please? I am trying to upload a video to dtube and it says i must upload a snap , i do that and it just keeps going round and round... any help please!!

very good post you i will try the way that post

I have a question about the local IPFS, the one an user can activate in order to host other user's videos to alleviate some of the gateway's traffic on DTube's end. Do you know how we can install it?
I have checked some tutorials, but the link they provided to install it is broken :)

·

I'll answer this!

Installing IPFS and managing files is hard for non-tech people from what I know, I personally only use the command line. Some people have succesfully been using that though : https://github.com/ipfs-shipyard/ipfs-desktop

However, with the work done on the uploader explained in this article, we plan to offer an easy-to-install software that will help you upload your videos and manage your ipfs pinned files, from your pc.

·
·

Thank you so much, I'll check it out once I go back home!

·
·

Can't wiat for that. I would probably offer some space/bandwith to seed videos (about 1TB, 10MB/s upload). Right now I use webTorrent but to be honest it works a bit weird :|

useful postings. I like your post .. thanks

I'am new be here, i want lear abouth dtube but i don't know how i can learn abouth this

·

You can read the about page of dtube!

Awesome. So cool to see how it's running. Lots of work goes into it and we appreciate it!

I got some setting from your article that's I'm just going to convert to before I upload.

Thank you for the details of this amazing work. I just got started with posting my videos and I would like to understand what video format you recommend uploading at (the original video), because I've tried avx since it has a much smaller footprint but then I lack the ability to take a snapshot since the upload window doesn't display the video itself. Maybe avx is not a good idea. Would love some pointers and thanks again for the fine work.

·

Like I write in this article, we recommend h264 video encoding and aac audio encoding

good, vollow & votes @wannis

Can anyone please help explain how to use dtube. Im new

·

See dtube about page ☺

DTube problem. on my iphone I can not click the search button on d.tube. the icon is there, but there is no function.

Hi there superkoala .. I just read ... Videos are limited to 20 minutes? That would explain why I actually cant upload a Video which is 30 mins long ... but it doesn`t explain how it was possible for me to upload a Video which is nearly 50 minutes long 2 days ago? I just wonder ... how could that happen ... by the way, I´m german ;-)

·

Hi, there are no limitation for the video source! But video source upper than 20 min will not be encode to 480p

·
·

Aaah ... okay :-) Now i got it :-) Thx for your answer!

If I join @Dtube or upload videos to @Dtube will I still retain full ownership of the video? Will I be able to delete the video if I choose to?

·

I think you can't delete it, but I'm not the best person to respond to that

Vote back

Hello @dtube, I'm from Aceh Indonesia, and I love your posts. Greetings know, and always be in luck! Please vote and follow me.

this information is very vital we need to let it go viral by resteem it's perfect

Are there going to be any updates to support higher frame rates or bit depths? I've defaulted to 30fps on all my videos now as none of the ones I uploaded at 60 frames will play without locking up. I noticed that streaming rates back to users are low as well. I've been trying to keep my videos at 4.5mb/s or less for now, but its hurts the quality fo the video too. I wish the platform would better support faster delivery of video streams to users and higher frame rates.

Either way, DTube is great for what it is, and its obvious you guys are working to make it even better over time!

I'm hoping that DTube can become a better alternative to platforms like YouTube over time as well. YouTube is a well functioning platform, but its useless for creators who intend to actually earn from their work. Google's ad revenues are now effectively totally dead! You can't make a living on YouTube any longer, hoping DTube will change this.

·

Thank you for your message. Things will be better soon. But video source can have framerate and bitrate you want, there is no limitation on it

·
·

Thanks again Super Koala, I totally honor the work you guys do here!

Thanks for all the hard work you guys are putting into this project, you guys are awesome @dtube.

Thanks for sharing @ superkoala 🤛🏾

·

Thank you!

Hello, I'm a dtube beginner. I hope to learn a lot from you about dtube.
My next hope is to buy the latest laptop, because my laptop is now broken, I use it for 6 years, the output of 2012

T_T

It would be interesting how the encoding process works so some can perform it on there machines and upload the product directly to Ipfs. I mean you could also put it on YouTube and download it afterwards but I don't want to feed YouTube with my videos.

·

Ok after thinking of my answer I could just read the documentation of ffmpeg and work on a script.

Dtube is great and i would really love to use it more but some competitions ive entered discourage using it due to issues. Even on some smartphones it seems videos just will not start.
Youtube is starting to be universaly hated across the board, from creators to users, and dtube really has an opportunity here and now.
I really hope you seize it. 😁

·

I hope too, we work hard on it!

Its very good ! and interesting :)))

Hey @superkoala,

I just uploaded my first music video: https://d.tube/#!/v/mewnusic/xsyh9662

The experience has been smooth; although I'm wondering how can I change my featured image?

Nevertheless, looking forward to the next enhancements of Dtube and hope you can check our music video out. We spent days of production time to deliver a good quality music video to the Steemit and Dtube community.

Enjoy!

·

Thank you! You can change it with steemit or busy if you can't with Dtube

·
·

Hi @superkoala are you on discord by any chance?

I believe that Dtube is next station for all vloggers Because of it's innovation and being easy to use :)
We were waiting this solution long time ago to be the best alternative for adsense as adsense is not transperant.

Great work Guys :D

Hey there. I think I found a bug in the display thingy of Dtube.
Untitled.png

Notice how the suggested videos on the right side are cut out. It seems like the whole thing's been zoomed in.

Thank you for the great work on DTube so far! I'm excited to see where this goes in the future!

Very nice excellent post! If you want to support me in my last publication, it would be a great help to my regards

kingtaunt.gif
rabbit_footer.gif

https://steemit.com/technology/@jesusj1/puede-la-tecnologia-blockchain-sobrevivir-sin-criptomonedas-jesusj1

thanks for great information
please vote my post https://steemit.com/dtube/@syukranpta/4sy9d5c9

Upvote me please

Thank you your amazing work.The video list is extracted from steem's blockchain. and each video.the next article wll explain these changes. I like it your post. Thank's for the sharing post.

it's amazing there's nothing to waste,
that's all I can say

Great job! d.tube is looking and working great. I hope d.tube is successful, maybe you can give youtube a run for it's money ;)

The only things I really missed when I recently visited dtube, iw a my subscriptions button, plus it seems that the search engine can't find channels by their name. Other than that it's super duper.

Hi d.tube can you please be able to allow mobile creators on iOS device to be able to upload videos to your platform Please and thank you

·

There will have some fix in safari soon

·
·

What is the best mobile iOS browser to use Firefox or Google Chrome?

I know this may not be the place to ask this but the contact page led me here. I was wondering about the earnings on the video. One of my videos says it's earned 0.07 in steem but I have no idea how to collect or if I even can. If anyone knows were it's best to ask these questions please let me know or if you can answer the question I have it be appreciated.

The platform is nice. I like D.tube more then youtube.But it's the same "be popular" think like in youtube. If i dont have friends i can't get even dollar:) But i will tray how best i can.

Thanks for your information

may i know how to join dtube member

Post yang bagus...
Tapi saya tidak pandai untuk membacanya...

hi - you are aware that you are on the HN front page now right ?

Please update FAQ regarding the 480p encoding

Great post! I'm STILL receiving an "Error while submitting to the blockchain" message when I try to upload. I'm using an mp4 file, jpg snap, 5 tags, standard title and description.

I've researched and followed just about every guideline I could find, yet I still get the same error. Do you have any insight into what could be the problem? I LOVE this platform and am dying to get the ball rolling. Any help would much appreciated!

·

I’ve just made a post on how to get through uploading error on d.tube.
check it out! @kurtjenkins @dtube
https://steemit.com/dtube/@blascobite/dtube-upload-error-and-how-to-get-through

Wow! I’m glad that DTube is in full force on improving the platform. I’m waiting for the day when DTube finally overpowers that giant bully called YouTube 😊

@superkoala why your post is always resteem by @Dtube i am also a @Dtube user he never resteem my post ever ?

·

@sharolyn it was the only one time when my first post was resteem by dtube, why you said always ? What do you mean ? Also my post explain how work the uploader and I am the developper of the uploader, and the uploader is a part of dtube

·
·

@superkoala i mean that what kind materiel @Dtube want to resteem i am new one here thats why i ask if you'll guide me it'll be your kind. it's really hard for me get upvoter even my materiel is not bad

·
·
·

Yes your post are not bad ! Dtube never resteem post, except those made by heimindanger and mine to explain what we do in Dtube because we are developper of Dtube

thanks who already understand now

Trying to find the proper place to ask this. When I try upload a video lately the progress gets stuck here,any advice on how to fix this?

dtube.png

Thanks! I look forward to the next article.

Thank you for the detailed post. With the IPFS daemon, how much file storage does it need to store all these videos? Does your team have any other storage nodes on the IPFS network seeding these files?

I'm pretty disappointed in D.Tube, I've tried about 10 times now to upload short introduction and it just refuses to work.

I have no problem uploading videos to Youtube, Bitchute, Minds.com etc but the D.Tube uploader will not work for me.

·

are you using Chrome? is it a mp4? I have a dislike for anything google but chrome seems to be the one that works for me.

·
·

I switched to using Firefox and the uploading seems to work better

·
·
·

Firefox is tricky, does not always allow a upload. Hang in there. I Still think this is a good investment. You should also try Busy.org. Nice interface and the videos play on the home page without embedding them. Good luck. Roberto

Thank you very much for making me feel secure while uploading. Most of the time I am uploading via a phone tether to laptop. I have been using Busy.org to combine steemit, steemit chat, dtube into a nice looking interface. Keep it coming. Plus this is an investment for me. Thanks again.

My dear sir i am Upvote you Upvote me back
thanks you

@superkoala i know this is quite old post but i would appreciate if you would take time and reply to my cooment.

i am quite new to steemit - joined end of May 2018 and i know only a little bit of dtube. i was really hoping to uload my videos there, and i come across error by erro,r it will just not let me do it.

i wonder if you can direct me to article/person who can help me? I am really keen to get there , but i am struggling on my own and can't find any information anywhere. and i think i've tried everything now!

i am hoping you will reply.

ps good luck with your account, i can see you are doing very well :)

Interesting info. One of the complaints I heard about this site is how long it takes to upload stuff. I'm sure the upload speeds will improve as this site gets more popular and stuff.
Hopefully this site can beat the living shit out of YouTube. Fuck YouTube.

·

Upload duration depend on client bandwith, but after, encoding process can be long due to queue. It will be better soon