Logo

Python port of Ark Core

Tue, Jan 22, 2019 12:53 PM | 5 Development - Software Submitted by Tomaz Sifrer
Estimated Work Hours
800
Funding
30000 USD
Status
Approved

Porting Ark Core code to Python language, to potentially increase community/developer reach and diversify core node code with another language.

Prerequisites

Requirements

A couple of development servers (up to 4 servers) to run testnet/devnet and Python Ark Core on.

Cost-benefit analysis

Money costs:
Servers: 4x server (4 vCPUs/ 4GB Ram) $40/mo each = $160/mo
Work: The rest of the cost is work.


Benefits:

- Broader developer access: people with different skill sets could start contributing new features to the Ark Core implementations written in the programming language they are comfortable with
- Running forging/relay nodes in different programming languages: can improve security if the bug affects only one implementation and not the other
- Attracting new people to Ark and cryptocurrencies, as a lot of schools are now teaching Python language as the language of choice
- Having hackathons/workshops for people new to programming/Ark/cryptocurrencies as Python is quite easy to learn and get your hands dirty with it
- Fresh pair of eyes auditing the JavaScript code


Potential costs:

- Bugs introduced only in one implementation of Ark Core could lead to a fork
- Split development of Ark Core between Python and JavaScript implementations
- New feature sync between Python and Javascript implementations of core
- Divide the delegate and dev community between Python Core and Javascript Core

Milestones

Design

Python port would have a modular design, very similar to JavaScript implementation. 3rd party “plugins” would hook into the core almost the same way as they do in JavaScript implementation, but the “packages” would be treated differently. The implementation of them would still be as a “plugin”, but they would be treated as a module of the core implementation that is optional to use (similar to Django implementation of the “contrib” package if anyone is familiar with it). The “packages” might also not be named 1to1 with the JavaScript packages as it might make more sense to combine them together in Python implementation (not sure about this one, will need to read more JavaScript code to see how everything is implemented)

Documentation will be mandatory for every function, for ease of understanding it and easier onboarding of new people. Documentation page would then be generated from the documentation in code using Sphinx and some other tools.

Testing

Source code will be tested with py.test and will try to have near 100% test coverage. Now bare in mind that the near 100% test coverage doesn’t mean anything if the tests are bad. The only way to truly get a good test coverage is to educate and encourage contributing people to write good tests and for core developers to properly review pull requests.

Launch Plan

Python port of Ark Core will launch in stages:


Stage 1: Get relay to work on devnet

This means porting over the following core packages:

- Core
- Core-blockchain
- Core-container
- Core-database
- Core-database-postgres
- Core-http-utils
- Core-event-emitter
- Core-p2p
- Core-transaction-pool
- Crypto (already implemented in python-crypto but might need modifications)

Development time: 10-13 weeks
Price: $20000
Payout request schedule (4 payments):

- after first 3 weeks (20% of the price): $4000
- after next 3 weeks (20% of the price): $4000
- after next 3 weeks (20% of the price): $4000
- after stage completion (40% of the price): $8000


Stage 2: Get forger to work on devnet

This means porting over the following packages:

- Core-forger

Development time: 4 weeks
Price: $6000
Payout request schedule (1 payment):

- after stage completion (100% of the price): $6000


Stage 3: Port over core-api

Development time: 3 weeks
Price: $4000
Payout request schedule (1 payment):

- after stage completion (100% of the price): $4000


Stage 4: Porting over other packages that are not mandatory for running the basic Ark Core - I’m hoping that here we start to get help of the community (or maybe even before this stage). This stage is excluded from this proposal as it’s a community effort and these packages are not mandatory to run relay, forger or api (the Ark Core).



In these stages, there are missing packages that I haven’t identified yet as needed for the specific step at the time of writing this proposal. There are probably a lot more things that need to be ported in stage 1 and 2 to get things to work.
All stages include extensive testing on testnet and devnet if applicable.
To make it so I could work on this full time and get it completed as soon as possible, I'd request for multiple payouts based on the schedule described under each of the stages. I’m expecting to get the payment in 1 week after the payout request which also gives you enough time to review each completed stage before finalizing the payment.
Development would be out in the open (on GitHub), so at any time, people could take a look at the progress I’ve made.

Support & Maintenance

I’m also hoping on the help of community contributions for the maintenance. I would also like to get a couple of maintainers with me on this project, so we can share the burden of reviewing contributions, developing new things and keeping the Python Core up to date with the JavaScript Core implementation. I am very eager to help support and maintain the project and will probably also try and run for delegacy.

Additional Information

Community Feedback

Community feedback was very positive and encouraging. A lot of community members want to see this to be a reality and a few of them also raised some of the concerns stated above.

Votes

Approval

There have been 3 votes for approval by outboard, Moon (biz_network), Del.
outboard Tue, Jan 22, 2019 1:11 PM

Major applause. I can't wait to see it and contribute. This is exactly the type of project where the ACF funds can have a dramatic impact.

Moon (biz_network) Sat, Jan 26, 2019 4:52 AM

Probably the best use of ACF funds to date.

Del Sun, Jan 27, 2019 8:05 PM

After reviewing your proposal and comments that were brought up as a result of it, I found your endeavor to be reasonable and I think you are capable of completing this task. I think that having more Python integration into Ark Core would enable a lot of new comers, who tend to learn Python as their first language, to be welcomed and expand their interests by working with our project.


This will be considered the largest project to be funded by the ACF and we expect a lot of transparency on your part as you begin this journey. Please make it your mission to comply with the requests of future ACF Members as they will determine if/when you get paid.


All in all, thank you for submitting your proposal and good luck!

Rejection

There have been 1 vote for rejection by cam.
cam Fri, Jan 25, 2019 1:11 AM

I apologize for the delay. I had to think long and hard about this one.
I am going to air out my thought process here as to why I am choosing to reject.


I'd like to mention that deadlock holds you in high regard and I have no doubt about your ability to be able to pull this off. I hold deadlock in high regard, so by proxy I have a great deal of respect for you and your ability to complete this project.
But, part of the job of being on the ACF board is to look at everything as objectively as possible.


I listened to a lot of input on the Slack the other day when this proposal came to light and had to weigh the input as objectively as possible from the community in a way to justify if funding this project truly would be in the best interest of Ark and the community.


Some of the pros that really stuck out to me were: Python is no doubt a growing language of choice, a handful of Ark Community Members use it, so it could be beneficial to have this as an option, students are also learning Python increasingly in schools as opposed to other languages. The price is absolutely fair for the per hour rate you're asking for. A vulnerability in the current Ark Core may not translate to Ark Py Core. Vulnerabilities may be discovered when porting. Your milestones were fair and make a lot of sense.


Some of the cons that stood out to me were: maintenance, I liked your answer that you were more than willing to maintain the new Ark Py Core for the foreseeable future, but, I am concerned that it could fall to the wayside. To delve into that a little further, I feel little patches here and there would not be a problem to maintain, but the big ones may be... and for every bit of time that Ark Py Core lagged behind it, the more that maintenance time/effort/resources is going to compound. This also speaks to a concern I have about security. For security patches, if a vulnerability were to be discovered, Ark Py Core would need to react just as quickly (given the severity) to avoid damage (given the circumstances and adoption). It was also pointed out that while Python is being adopted more, one individual mentioned that core being in js never appeared to pose an issue or a concern at hackathons (in their experience, a bit anecdotal, but its really all we have to go off of). There are plenty of SDKs to interact with Ark, Python being one of them. The big one that kind of sealed the deal for me was this: is there a need for it? I don't exactly have the answer to that. I haven't heard that there is a need, and to be candid we aren't bombarded with projects or inquiries (at least not in the public eye) asking for this. Yes, we'd be setting ourselves up for the future of the 'if' we needed it, but I don't see a demand right now.


A couple takeaways and side notes:
I think if data was provided that there was a demand for this my tune would be a bit different. If that could somehow be provided then I'd reconsider approval.
If this proposal doesn't get approved by this board... there is a new board coming into tenure within a couple weeks. They may feel differently, I'd definitely encourage you to resubmit the proposal to them.
If it is still not approved, perhaps in the future a demand will present itself, I'd encourage you to keep this proposal in mind. Perhaps it is the right proposal but not the right time.
In the past, different members have created things and then asked for funding and it has typically worked in their favor. I know nobody likes working for free but it may behoove you to initiate the work and then seek funding, totally up to you.
This was a big risk v reward struggle with me. Unfortunately the risk outweighed the potential reward due to my aforementioned demand concern.


I wish you the best of luck and I hope you still remain part of the Ark Community. :) I look forward to seeing you around.

Leave a Comment

Comments

Math
Tue, Jan 22, 2019 1:17 PM

Available Funding: $26,927.00 Funds asked: $30,000.00

Moon
Tue, Jan 22, 2019 2:10 PM

GitHub?

Tomaz Sifrer
Tue, Jan 22, 2019 2:20 PM

@Moon My github is: https://github.com/tsifrer

outboard
Tue, Jan 22, 2019 3:23 PM

@Math - We have another ~130,000 ARK sitting in voting wallets, some of which would be pulled in for this project.

Mike
Sun, Jan 27, 2019 8:01 AM

This would be great for python coders who want to be able to tweak and experiment with Ark core. Python has been by far the most popular language at the hackathons. A python port would help encourage academic and scientific use of Ark. I hope this gets approved.