Monero Dev Meeting Note Highlights – 2017-03-26
Logs
351 lines
fluffypony2. Brief review of what's been completed since the previous meeting
fluffyponyok so since the last meeting I guess the main thing is we tagged and released 0.10.3
fluffyponywhich we're about to deprecate
fluffyponylol
fluffyponyare there any issues with 0.10.3 besides the cumulative block size thing?
fluffyponynow's the time to point them out
hycno idea, but I'm running a build from a couple days ago
Jaqueeme too. no issues so far
johnalanBeen running on OSX since yesterday. No issue.
fluffyponymoneromooo: any idea why the issue seems to affect so few?
hundehausenSmart Mining is not working for me on newest macOS
moneromoooDunno. Low level processor specifics I guess, but… shrug.
fluffyponyhundehausen: it only works on Linux
fluffyponynot on anything else
samsunggalaxyplayerI have smart mining running on Windows right now
Jaqueeyeah. windows + linux iirc
Jaqueebut not osx i think
luigi1112lunch time
fluffyponyI like to pretend that Windows doesn't exist
fluffypony:-P
Jaqueelol
moneromoooWhat is it ?
fluffyponymoneromooo: you open them to let air in
moneromoooAh, doors.
hycusually lets bugs in too
amiuhlesmaller sized doors basically
gingeropolousdrumroll
fluffyponylol hyc
btcltcxmrmaximalwinblows sucks
hycwindows and doors in Ireland have no screens. I dunno what's with these people.
fluffyponyanyhoo
fluffyponylet's move on
fluffypony3. Discussion of fireice-uk's proposal (as started in #1828
fluffyponyI'd like to move this to Funding Required
fluffyponyand fireice-uk updated the funding costs based on current pricing
fluffyponyobviously there are some consensus-critical aspects to it, so I think it's worth discussing
moneromoooWasn't this a wallet thing ?
btcltcxmrmaximalhttps://github.com/monero-project/monero/issues/1828
xmrericYes. Speedup on Intel/AMD processors, which is helpful considering RingCT has slowed sync down.
fireice-ukit is a wallet thing (unless you want to use it somewhere else)
hycringCT has slowed wallet sync?
fluffyponymoneromooo: if we replace SUPERCOP then it's consensus critical
vtnerdI don't see how ringct slowed down wallet sync … ?
moneromoooThen no consensus issue. And if it proves good for a while, then it can be used in consensus.
pigeonsxmreric: how has ringct slowed down sync?
xmrericI thought I had heard that from others
vtnerdthe additional work comes when a output match is found
fluffyponyso I guess wallets with thousands and thousands of ringct outputs?
fluffyponyxmreric: that's daemon, not wallet
fluffypony1828 is a proposal for a wallet change
xmrericok
vtnerdits more work on the node verifying the block, but not the wallet since its not reading it. I suppose there is some additional time for transmission/marshalling/unmarshalling, but this is smaller than any crypto
moneromoooThe bottleneck's the daemon anyway.
btcltcxmrmaximaldaemon sync time seems a lot more important than wallet sync time (in comparison) if our primary goal was to encourage more full nodes.
vertpUnless you're using a remote node, no?
hycthis complicates the build if we want a crypto/ subtree just for wallet and one just for daemon
moneromoooHmm, fair point.
fluffyponyok so then here's a suggestion
Jaqueethe amount of tx's/day is higher since the date around ringct was activated. So wallet sync slows down. but not really related to ringct
fluffyponywhat if we had cryptoopsbuilder run on build
fireice-ukbuild won't be more more complicated - just more symbols
fluffyponyand use the existing stuff by default, but optionally use the newer SUPERCOP / whatever
fireice-ukmy suggestion would be to use ge64* symbols for the new code
moneromoooBTW, is that not what you wanted to replace by… tweetnacl or whatver it was ?
fluffyponymoneromooo: yes
fluffyponybut only when TweetNaCl has finished formal verification
moneromoooAnd that proposal replaces it with this, or another replacement ?
moneromoooAlright…
hycI'd prefer we just keep the generated code statically committed to git
hycno idea what environment the builder might break on
fluffyponyhyc: the builder is pretty simple (just splicing text really), but it does add a python dep to the build process
hycyeah, let's not do that.
moneromooofireice-uk: did you try running a wallet refresh without any crypto to see how much faster it was at best possible gain ? IIRC, my bottleneck is the daemon (SSD, though CoW fs).
fluffyponyeither way, in the long run I'd like to have a default "safe" crypto implementation, and an optional fast one
moneromooos/without any crypto/with the actual tx scanning disabled/
fireice-ukmoneromoo: bottleneck is the poor fetching from the daemon
moneromoooSo changing the crypto won't do a thing right now, right ?
fireice-ukit somehow mananges not to max anything
fireice-ukthat is the part 2
fireice-ukcrypto is part 1
pigeonsso likely this is a premature optimization for now
fireice-ukpigeons: want to swap round the order?
vertpwhy not swap the orders fireice-uk? And do daemon fetching optimization
fluffyponynot a bad idea
moneromoooOh ok. There are two things that should be easy win there: store non prunable separately, and maybe fetch a bunch of them at once (wallet refresh always has pretty much N..N+dN txes).
gingeropolousso part 2 is the actual optimization? and part 1 is… ?
moneromoooI wanted to do the daemon thing for a while, but looks like I won't have to :D
fireice-ukgingeropolous: part 1 is crypto optimziation, part 2 is parallelism opt
hycbut current discussion says crypto opt will be overshadowed by daemon
fluffyponyok so we swap them around and do part 2 first, and then revisit how to structure part 1 after that?
hycmakes sense
fluffyponyfireice-uk: that sound ok?
moneromoooWell, that's my recollection of my particular machine anyway. Might differ for others.
fireice-ukmy suggestion would be to do part 1 first - this way you can have a loot at it before merging
fireice-uk*look
moneromoooI want to have a look at 2 also before merging.
hyclol
vertplol.
fireice-ukof course, but i assume 1 will require more time
fluffyponylol
vertpbut if its has more immediate benefits, why not go with 2 first?
hycyeah it's sounding like 1's benefits will be unmeasurable for now
vertpit makes even more sense to do part 2 first if it is less complex/faster to implement.
Jaquee+1
moneromoooYes, do the easy wins first, and the possibly dangerous stuff might not be needed (and will only work on x8664 anyway AIUI).
fireice-ukok, that's fine with me
moneromoooKeeping in mind you also need the full blocks to serve syncing peers.
vertpgreat!
fluffyponyok cool - I'll move the proposal to funding after the 0.10.3.1 tag
samsunggalaxyplayerso in like 3 months /s
fluffyponyhah hah
fluffyponytouché
vertpSince we are on a similar topic, could I bring up ZMQ? That should also speed up sync time/provide faster wallet func at some point no?
tewingetnah, fluffypony doesn't run on tewinget Time™
fluffyponyvertp: it should provide better scalability for multiple wallets hitting the same daemon
fluffyponybut I don't think it'll provide speed benefits beyond that
vertpso light-wallets. Is there anything new on that front tewinget.
tewingetagreed
tewingetwell, I've been working on merging from upstream this morning
tewingetI think I've just got it sorted
hycyay
vertpwoo!
gingeropolous!!!!
johnalanNice
fluffyponylet's stick to the schedule plx
tewingetfew things changed in core that threw wrenches in the merge >>
tewingetsorry fluffypony :)
vertpyes, sorry.
fluffyponyok so
fluffypony4. Remote nodes (ie. a discussion of #605)
moneromoooWell, I was thinking about this, and I will do a wallet mode where a full wallet (ie, phone) can connect to a view wallet (ie, home server), and update from it. That should be super fast.
fluffyponymoneromooo: that's exactly what vtnerd is doing
moneromoootewinget: I think I kinda added a new RPC… a few days ago…
fluffyponyso would be duplication of work
moneromoooOh, OK.
johnalanWon't that only show incoming though?
fluffyponybut let's back up a second
hycso a wallet can sync from another wallet?
fluffyponybecause I think that maybe there's some value in the idea of 605
moneromoooIn my idea, yes (really, transfer output data).
fluffyponybut the specifics aren't great
fluffyponyfor eg.
moneromoooBut I dunno what vtnerd is doing.
xmrericLots of GUI users want this on some level or another.
xmrericI'm pretty big on emphasizing away from using remote nodes as best-practice.
fluffyponywhat if an unsynced daemon, when it has a wallet client requesting outputs from a certain height, picks a random peer and asks that peer for the data
xmrericBut for people in developing nations, etc it's a good option to offer
samsunggalaxyplayerI know that we all want everyone to run a full node, but I imagine less than half actually will, and that percentage will only decrease over time with new, non-technical users
fluffyponyie. without range proofs / sigs / etc.
gingeropolousthe random peer has its rpc open?
fluffyponythe peer could lie, but the node will eventually know that it has
fluffyponygingeropolous: no
fluffyponywe don't need RPC for this, we're already talking to the peer using the p2p protocol
moneromoooWhy would the wallet request outputs for a given height, if the daemon isn't synced to that yet ?
fluffyponymoneromooo: restored wallet, or loading a wallet file
Jaqueeif the daemon isnt synced it shouldnt be used by the wallet
fluffyponyor creating a new wallet
moneromoooRestored wallet would not, it has no idea about where it has outputs.
fluffyponymoneromooo: restored from seed has a hardcoded restore height
guzziIt is similar to how hadoop works
moneromoooNew wallet wouldn't either. They'd get that info from the daemon, who'd necessarily be synced up to that point.
guzziHere is the first answer it may be wrong
moneromoooUnless you delete your blockchain after the wallet learns abvout those. But then, your problem.
fluffyponymoneromooo: in each of the instances we either have a block height or we have a date that we can correlate
moneromoooOh, you want all outputs ?
fluffyponyfrom that height or date, yes
guzziI like it. The attack would bre someone setting up a ton of fake nodes.
fluffyponybasically have the daemon tunnel "remote node" functionality to a peer
moneromoooOK, so essentially, syncing the chain with no vcerification whatsoever.
fluffyponymoneromooo: yes - "pre-syncing" it
fluffyponybecause the node will catch up, and then the wallet will know if outputs have been withheld
moneromoooI actually had an idea about this a few days ago, where you could sync to a daily set of key images and outputs. Daily, verifying nodes hash it into the blockchain.
guzziBasically a no lock read
moneromoooSo you can sync to that, check hash, then sync the last day's chain on top.
gingeropolousbut this would leave the wallet in a state where it can't create transactions until proper sync'd?
fluffyponymoneromooo: the problem with that model are the oracles
moneromoooIt does require some trust, though.
moneromoooGo on ?
pigeonsso the tricky part is the rules to make the block invalid if the miner lies
gingeropolouser, until the daemon is proper syncd?
fluffyponygingeropolous: the daemon could also tunnel requests for ring outputs or whatever
fluffyponythe trust model is the same as using some random guy's remote node
Jaqueegood idea. but still doesnt help users without enough disk or those on a slow connection
guzziTrue
samsunggalaxyplayerThat's what I proposed #602 for
samsunggalaxyplayerPeople who do not want to run a full node
peroi had a kind of mostly trustless idea for this
fluffyponypeople that don't want to run a full node at all have to then use something like the MyMonero apps using the MyMonero backend instead of their own
fluffyponyor Exodus or Coinomi or whatever else exists
perothat would poll multiple nodes for the same outputs and verify them
peroand store those locally
fluffyponypero: that's even worse than a remote node
perofluffypony: +1
perosees himself out
guzziOr this and never sync tge daemon
guzziLol
rehrarWhat if the time connected to a remote node is limited? Just setting up the GUI it connects to a remote node, and they can use right away, while stuff is going on in the background? (Sorry, just an idea I had. Not a developer so I don't know if possible)
fluffyponythe larger issue here is that we can't do something like SPV
fluffyponyso we really have nothing between "run your own node" and "use a centralised service"
rehrarStuff = syncing
ArticMineShould we looking at people connecting to their own remote node form say a mobile device?
fluffyponyrehrar: that's exactly what I'm suggesting, but let the daemon "tunnel" the requests through
fluffyponyArticMine: the model here is people who don't want to run a node at all, not at home, not on a VPS, not at all
Jaqueerehrar: that's what #605 does
fluffyponywe already have a solution for people who are willing to run a node
rehrarSorry. A lot of the tech is going above my head to catch it all. :)
moneromoooWell, they can use paypal, and come back in 5 years.
rehrarTech talk
Jaquee#605 connects to a remote node while local node is syncing
jacobjewelerI agree, fluffy. I think the real issue is people not having enough knwoledge to install nodes. An installer on windows and .deb in apt would increase full nodes immensly.
gingeropolousi like it. the pre-sync idea. using the daemon. it opens up the whole network as a source of remote nodes, which decentralizes the effort
xmrericWhat if all this work gets done, but then this audience just uses web/mobile wallets anyways
pigeonskeep improving the ability for people to run their own node before making it easier for people to use a different model
gingeropolousand because the daemon is running
gingeropolousit will be synchronizing its own copy of the blockchain
fluffypony@xmreric that's the most likely outcome
hycthat is trickier
fluffyponypeople are going to use MyMonero / Exodus / Coinomi even if we have a magical remote node model that doesn't vampire the network
guzziIt could even hold part of the chain and randomly ask for missing parts
hycblock data sync'd this way will need to be stored differently than from regular syncing
pigeonspeopl are going to use worse options than those even
fluffyponyhyc: agreed
fluffyponypigeons: store on an exchange :-P
samsunggalaxyplayerI like the pre-sync as well. But until we have MyMonero/Edodus/Coinomi, people will use a remote node in an inefficient way
fluffypony@samsunggalaxyplayer then let's not make it easier by having a drop-down
gingeropolous^
Jaqueewhy not make it easy while waiting for a better solution?
fluffyponyremember that a lot of decisions we make today, we're stuck with for 5+ years
gingeropolousthe effort wall to hack the system to use a remote node isn't that steep anyway
fluffyponyJaquee: because ^^
fluffyponypeople become reliant on quick fixes
hycSo somewhere we need the doc to say "you must have a computer with at least xx GB of disk space that you are willing to leave running 24/7"
tewingetknows this, and as such leaves most decision making to fluffypony so he can be blamed in 5 years.
moneromoooAh, tewinget the Wise.
endogenicin 2023
Jaqueei just don't think we should be holding back on UX just because we don't have a better solution yet
pigeonshave a nice message, now that you have verified the blockchain, we notice you have been screwed, pick a better node next time
fluffyponylol
moneromoooAnyway, this has turned to a disparate set of confusing stuff now.
guzziLol
fluffyponyJaquee: the GUI is meant to operate with a full node that you operate, it's not a lightweight GUI
ArticMineDo we want to encourage people connecting to a untrusted random node
fluffyponyArticMine: no we don't
samsunggalaxyplayerSummary: for smart syncing with fluffy's "pre-sync" approach, against anything that makes using a remote node easier
guzziFor ppl who want to use a phone could it never sync?
cryptocomiconsounds like we need a monero node appliance, like the wifi router that everyone has in their house / flat
samsunggalaxyplayerI agree #602 is a short-term solution. I think it's better than telling people to go to MoneroWorld to get a random node, but if we have a better solution going forward, that's preferable
johnalanGood idea
hycyes but wifi routers tend to be 32bit
jacobjewelerThats why i think installer for windows and adding .deb to apt repositories will have it so people can be guided through an install and proper installation can be verified
guzziAnd always use a random?
fluffyponyguzzi: phone would be MyMonero + your own node / MyMonero backend OR Exodus OR Coinomi
gingeropolousand they're not cheap
guzziOk thanks
jacobjewelerAdoption rate will increase full node usage
fluffypony@JacobJeweler we're definitely working on improving that with the GUI
ArticMineI say make it easy for people to set up their own node to connect to. Appliance like
guzziThat is horse poop
fluffyponywith the auto-update thing
peroperhaps there's room for an unofficial gui fork
guzziNo way ppl walñnt full nodes
fluffyponygingeropolous: http://imgur.com/a/3mMBE
gingeropolousin my mind the remote node thing has 2 components: 1) instant on 2) no blockchain storage.
fluffyponygingeropolous: pigeons is working on it
hycright now the only way I see to make this easy is with kovri, so we can ignore firewall and port forwarding issues
fluffyponyhyc: +1
gingeropolouswe can address instant on with lots of things
gingeropolouswe can't address no blockchain storage. And those that don't want to store the blockchain will always use some lighter weight thing, so … i think im rambling.
jacobjewelerGood point hyc!
amiuhleHow about creating SD card images with the blockchain preloaded for a specific monerod release. You'd "just" have to download the image, flash it and start up monerod
johnalan+1 hyc
amiuhleah, like for the Pine64 or something similar
hycpretty big downloads. they don't compress well at all.
hyc13GB now
Jaqueeand pine64 is to slow
johnalanDoes it even have native AES?
fluffyponypine64 isn't too slow?
amiuhlehyc: didn't you say you're running your full node fine on yours?
hycpine64 yes
peropine64 can run a node
hycyeah pine64 works ok as a fullnode. buy an expensive microSD
fluffyponyalright
tewingetI'll brb in like 10 min, fyi
fluffyponylet's move on
fluffypony5. Code + ticket discussion / Q & A
Jaqueeok. so #605 will not be merged?
fluffypony(we can carry on discussing this after the Kovri meeting)
Jaqueeall right
fluffyponyJaquee: no not with the drop down
hycyou can release the bootleg edition
fluffyponylol
amiuhleI've gotten the impression that there should be more unit tests from reading the last dev meeting
fluffyponyamiuhle: yes
jacobjewelerLets be honest, most users have windows. And harddisks that can easily fit the lmdb database. If you want great adoption and more full nodes on the network (people installing and usong their local node for gui/cli). Thats where the focus on something like an installer should be at.
amiuhlewhat if we at least request tests for new PRs?
moneromoooThen you won't get PRs.
vtnerdthat would help, but could be frustrating in a few components
fluffyponyamiuhle: we don't want PRs from new contributors mired in a list of things-the-PR-must-have
vtnerdfor instance, I added some to epee::stringtools, but those are isolated functions so its easy to setup the test env
hycsome of this stuff won't be easily detected in tests anyway. race condition with mining blocks, etc.
fireice-ukone more thing regarding wallet2.cpp
jacobjewelerSorry am on phone typing, slow to respond.
vtnerdhyc: yup. but figuring out a base framework for some areas might be helpful to get a baseline. but its decent chunk of work
fireice-ukit is already at an unwieldy 5kloc, what's the opinion on splitting it into smaller parts?
hyccan't detect some of these with code coverage testing either. code cov can't tell you about logic you're missing.
moneromoooTests can be added after the fact btw.
moneromoooIs there a split that makes sense ?
moneromoooAnd 5k is wieldy for any sane editor.
fireice-ukmoneomoo: takes 2gb+ to compile
hycyeah I wouldn't worry about wallet2.cpp at the moment.
vtnerdmooo Im guessing vim ?
hyc2GB+ to compile comes from all the boost headers and shit
moneromoooWhat I want is avoiding spamming the git log, as I use it a lot.
fireice-ukok
moneromooovim works fine with 5k, but most other editors are also not shit.
vtnerdfireice-uk: thats partially coming from the epee headers though, but some split may help a bit
moneromoooOr… I assume. 5k is not much.
hycI've done the experiment before.
fluffyponybbedit handles it fine
anonimalcoughs
hycsplitting the file up to try to fit it under 2GB
fluffyponyok guys
fluffyponyKovri meeting
hycmade no diff. it's the headers, not the cpp source
fluffyponynext meeting in 2 weeks, no time for Q&A, thanks for coming
Sources and notes
- Meeting log: Overview and Logs for the Dev Meeting Held on 2017-03-26, getmonero.org. The official post carried the raw log and pointed readers to Hello Monero for the overview.
- Original Hello Monero URL slug recovered from the Wayback Machine index; the original page content did not survive, so this reconstruction carries the log as archived by the Monero Project.