Proposal: Public Hivemind SQL service

avatar
(Edited)

Hivemind, a tool by Steemit inc., is a backend API service currently replace some of the endpoints of steemd. It's in use for a good amount time.

I've been running a public Hivemind node, distributing daily database snapshots almost 9 months. There is also tower, a REST interface for Hivemind data, is developed and maintained by me, so far.

RPC and REST interfaces are just the tip of the iceberg since the underlying database -PostgreSQL- offers much more flexible queries. (Check some examples)

I want to offer read-only public accounts to query the Hivemind database. This requires new infrastructure planning. Can't offer public accounts on the master PostgreSQL instance due to obvious reasons, I need new at least one or two replica instances where people can query and benefit the power of the Hivemind.

These come with additional development/maintenance/server costs. While I am happy to do that with my witness earnings, I already run a lot of dedicated servers.

Didn't really do the math, but probably get into a loss if we compare the costs with witness earnings. Here is my server list for STEEM related things:

  • Two dedicated servers for witness
  • One dedicated server for Hivemind
  • One medium-sized virtual private server dPoll
  • One small-sized virtual private server for scripts (witness backup failover, etc.)

TL;DR What will be delivered?

A reliable, fast, public SQL service (direct connection) of the Hivemind. I expect to deliver it before 2020. The plan is exposing Hivemind's database directly to the public with a good, old PostgreSQL user. (read-only)

Timeframe of the funding

1 year

Daily request amount of SBD

12 SBD

Q.A

  • What will happen when the funding window is closed? Will you shut down the service?

1 year is a lot, I cannot promise anything at this time. But, worth to mention, I already served free public Hivemind node for 9 months, and I expect to keep this service is up as long as I can, financially.

  • Will you cash out the funding instantly?

My plan is to power up the SBD, instantly after converting to STEEM. I will cover the server costs with fiat and don't create sell pressure on STEEM with the steem dao funds. However, this might change in upcoming months depending on the costs, my financial status, etc.

  • Are you eligible for that kind of thing?

I am a software engineer with 10+ years of experience. Also, experienced with Python (the language powers Hivemind), and PostgreSQL. (My github account @emre might be a good indicator about what can I do.)

  • What will be the outcome for community?

This is useful mostly for software developers, data people. Applications, dapps, reports can benefit this direct SQL access.

  • Difference between SteemSQL?

SteemSQL is a general blockchain data service, while Hivemind only stores a partition of the blockchain data. Mostly social part like posts, accounts, votes, relationships (follow, ignore, resteem), etc. It's also planned to store communitiesdata at Hivemind.

Voting for the proposal

You can view all the proposals on:

If you have any questions or suggestions, shoot in the comments section.

sps hivemind
0
0
0.000
58 comments
avatar

On steem?

Self proclaimed community watchdogs are actually hacking and stalking people.

This group https://steemit.com/@steemspeak is ran by https://steemit.com/@fyrstikken, They are hacking people that enter this discord server https://discordapp.com/invite/sqxV63P . Once you are hacked they profile you long enough to know all of your activities. They introduce workplace gangstalking visits and bring it to your attention cleverly and discretly so only you get it. They are very covert and tricky with everthing they say and do in this server. Some claim to have worked for or are working with the FBI and the NSA. Mixed with decomisioned traders and coding criminals, Its a scary mix of creepy people and thier intent is to drive you crazy and deplete everything you have ever aquired in life, even your freedom.

My hacking started with them introducing ransomeware that has options, dump my coin or delete my system drive. They acnowledged this to me in the server as it happened.

This hacking and stalking is done so they can manipulate the target to pump and dump for them. Threating and Making victims wear head phones, so nobody in your household can hear thier dramitization towards you. In 2017 this server was sidemarked as a drama show for entertainment purposes only as en excuse to say whatever they want all the way down to killing a politician. They have a server side command called "hey asshole" making a promt come up instructing everyone to wear headphones. They want everyone wearing headphones so nobody in your vicinity can hear them mess with you. Fystikken says its because of "mic feedback" but once your a target you get exactly why they make eveyone wear headphones.

With headphones on they subliminally direct victims with very low suggestive whispers mixed into their radio shows music. Combine this with them being able to manipulate their own individual outputs sound volume, which they control on the server side, they can pick what individual people hear and what at any given time. They decide who you hear on the server and change the volume controls for them. They can make some really low or off for those trying to investigate. They control who hears what and know who is who.

This team will find absolutely everything they can about you prior to using thier sugjestive and threatning program. They Introduce pictures that mean something only to the target, inducing paranoia while they watch and listen thru all of your circumvented devices. They are very aware of what makes you tick. They know your work schedule and use it for work stalking along side of this. I received multiple threating letters to my work and home addressed to my screen name from this group.

Fyrstikken tells people to sell their houses to buy crypto then dumps it on them. He has done many pump and dump scams. Quatloo being one of the first I witnessed and was done multiple times thru 2016 and 2017 them. It should be investigated for fraud. https://bitcointalk.org/index.php?topic=655793.0

0
0
0.000
Reply
avatar

Is this similar to the @arcange database but for PostgreSQL?

Posted using Partiko iOS

0
0
0.000
Reply
avatar
(Edited)

Yes, it will be the same except Hivemind doesn't store whole blockchain data like SteemSQL. Instead, it focuses on social aspects like votes, comments, posts, resteems, relationships. (ignore, follow, etc.)

In the future, communities data is also planned to store at Hivemind.

0
0
0.000
Reply
avatar

Sounds great! Looking forward to be one of the first to test :) You already have my witness vote and I will support this proposal.

0
0
0.000
Reply
avatar
(Edited)
0
0
0.000
Reply
avatar

It is available with a subscription fee.

0
0
0.000
Reply
avatar
(Edited)
0
0
0.000
Reply
avatar

Post includes a Q/A section explains the main difference.

0
0
0.000
Reply
avatar
(Edited)
0
0
0.000
Reply
avatar
(Edited)

:)

Then just don’t vote. You’re twisting words, I really dont have time for that usual wehmoen toxicity.

0
0
0.000
Reply
avatar
(Edited)
0
0
0.000
Reply
avatar
(Edited)

SteemSQL's and Hivemind's database are different. Hivemind focuses on the social/non consensus aspects more, and normalizes data in most of the places while SteemSQL stores all blockchain in a raw way.

So, saying that service will be a clone will be unfair to both projects because they're pretty different.

Can SteemSQL do everything that Hivemind can do? I don't know. I've never bought a STEEMSQL subscription.

I just ask why you want the community to fund a service that does technically already exist without community funding

Technically there is no service exists to expose Hivemind's database as several times, this post and the comments outlined. Since it's powerful database, it would be good
for developers, data science people to develop apps, explore the blockchain data, creating reports, etc.

You have an alternative with subscription model? Great. Use it, and don't vote for proposal, but for god's sake do not waste my time. You're muted on my list, and you won't get any answers from my side, anymore.

0
0
0.000
Reply
avatar

SteemSQL isn’t exactly raw. But it isn’t as focused on interpreting consensus as Hivemind. I think both have their uses.

0
0
0.000
Reply
avatar

I have used SteemSQL, it is awesome. However, it is not affordable for an average user, especially with the current Steem prices.

Building a free alternative for users and independent developers should be welcomed.

0
0
0.000
Reply
avatar

I run my own hivemind node as well as pay for SteemSQL access. I find that hivemind is often better for querying certain things as compared to SteemSQL.

In some cases, results come back faster in hivemind because of how the indices are arranged.

One of the major benefits of hivemind is how tags are tracked. If you’re doing content specific queries, it’s usually better to use hivemind.

0
0
0.000
Reply
avatar

Both Hivemind and SteemSQL interpret the blockchain differently. A really good example of how some Hivemind queries are useful over SteemSQL is doing tag queries. With Hivemind, you can do a query that matches posts that have all tags specified. With SteemSQL, this is difficult.

When I say "all tags specified" I mean that you want posts that match "tag a" and "tag b" as opposed to "tag a" or "tag b".

This is possible in Hivemind because tags are a dedicated table, so doing that kind of query is somewhat trivial. With SteemSQL, you could probably achieve it with sub-selects and temporary tables, but I haven't needed to look into it.

Some more examples: Hivemind Queries

0
0
0.000
Reply
avatar

Please explain to the average guy like me who has no idea of what your saying

0
0
0.000
Reply
avatar

Outcome of this project is that software developers, app owners, data people can have access to blockchain data via something called SQL.

SQL helps to query the data in a practical way in comparison to what we have in public resources.

0
0
0.000
Reply
avatar

So you wanna implement it on Steemit?

0
0
0.000
Reply
avatar

Not sure if I understand the question :)

0
0
0.000
Reply
avatar

I have voted for your proposal.

What kind of hardware resources would I need if I wanted to run my own copy of Hivemind/Tower?

Thanks for all the hard work you do.

0
0
0.000
Reply
avatar

I would suggest a dedicated server with at least 16gb ram and 256gb SSD. For me, I run it at a box with 32 gb ram and nvme.

0
0
0.000
Reply
avatar

Your proposal is lacking... to 99% of us.
It seems to be a proposal written directly for developers... and maybe that's smart because maybe you're only here to get the votes of whales and developers but you could at least tell the rest of us why we'd care about SQL hivemind and how it will impact our day to day activities... or don't because our votes aren't gonna really get you anywhere and maybe you only care about the votes of those who know the value of this database you're talking about.

Anyway thought you may want to know what most of us think about these proposals. We want to go through and vote on stuff but then we see that the proposers just don't care to write a post directed at us. But then again they're all being done by developers and I mean that's not their strong suite i guess. So i mean I'm not castigating you (you're doing what you need to do) i'm just saying it just sucks for us.

0
0
0.000
Reply
avatar

You have only me to blame for bringing this post to the trending.
Sorry if the post didn't include an explanation for non-dev people.

To sum it up, This lower the barrier of entry for developers to start experimenting. I don't see it as subsidizing resources but more as a playground to later develop more serious stuff that won't depend on this service.

0
0
0.000
Reply
avatar

That’s a good feedback, I’ll try to explain these stuff better if I create another proposal in the future.

0
0
0.000
Reply
avatar

Güzel bi aksiyon Üstad, eline sağlık. Bütün oylar senin olsun.
Keşke yazılımcılardan parti kuran çıksa da siyasi oylarımızı da ona versek ;)

0
0
0.000
Reply
avatar

Selamun Aleyküm Katılıyorum sisteme yeni katildim ancak boyle basarili turk kardeslerimi az goruyorum birbirimize destek ciksak daha iyi olur diye düşünüyorum. Başarilarinin devamini dilerim.

0
0
0.000
Reply
avatar

On steem, have crypto?

Self proclaimed community watchdogs are actually hacking and stalking people.

This group https://steemit.com/@steemspeak is ran by https://steemit.com/@fyrstikken, They are hacking people that enter this discord server https://discordapp.com/invite/sqxV63P . Once you are hacked they profile you long enough to know all of your activities. They introduce workplace gangstalking visits and bring it to your attention cleverly and discretely so only you get it. They are very covert and tricky with everything they say and do in this server. Some claim to have worked for or are working with the FBI and the NSA. Mixed with decommissioned traders and coding criminals, Its a scary mix of creepy people and their intent is to drive you crazy and deplete everything you have ever acquired in life, even your freedom.

My hacking started with them introducing ransomware that has options, dump my coin or delete my system drive. They acknowledged this to me in the server as it happened.

This hacking and stalking is done so they can manipulate the target to pump and dump for them. Threatening and Making victims wear head phones, so nobody in your household can hear thier dramatization towards you. In 2017 this server was side marked as a drama show for entertainment purposes only as an excuse to say whatever they want all the way down to killing a politician. They have a server side command called "hey asshole" making a prompt come up instructing everyone to wear headphones. They want everyone wearing headphones so nobody in your vicinity can hear them mess with you. Fystikken says its because of "mic feedback" but once your a target you get exactly why they make everyone wear headphones.

With headphones on they subliminally direct victims with very low suggestive whispers mixed into their radio shows music. Combine this with them being able to manipulate their own individual outputs sound volume, which they control on the server side, they can pick what individual people hear at any given time. They decide who you hear on the server and change the volume controls for them. They can make some really low or off for those trying to investigate. They control who hears what and know who is who.

This team will find absolutely everything they can about you prior to using their suggestive and threatening program. They Introduce pictures that mean something only to the target, inducing paranoia while they watch and listen thru all of your circumvented devices. They are very aware of what makes you tick. They know your work schedule and use it for work stalking along side of this. I received multiple threatening letters to my work and home addressed to my screen name from this group.

Fyrstikken tells people they are drafted like they did with Facebook at the beginning. He says Facebook was started by forced labor and V2K controlled slaves, just like he creates with his gang for steem and crypto. "Get to work bitch" he tells people once circumvented and intimidated.

If your an agent Investigating them, guess what? You are circumvented. They are high tech, over funded and well connected.

0
0
0.000
Reply