Skip to content

Commit

Permalink
scrooge/finagle: Remove generated HKT, FutureIface and their friends
Browse files Browse the repository at this point in the history
Problem

FutureIface is deprecated.
Scrooge is more powerful when used with finagle integration,  and in that case, higher-kinded-types won't play its advantages.
Scrooge generated code is growing which increases source compilation time, we should get rid of the unused parts.

Solution

Remove them from code-gen.
In generated code, replace HKT/FutureIface with MethodPerEndpoint.
Remove some deprecated builders in both scrooge-gen and finagle that require HKT as the parameter.

Differential Revision: https://phabricator.twitter.biz/D747744
  • Loading branch information
yufangong authored and jenkins committed Sep 30, 2021
1 parent 1f87b5e commit 8d768ca
Show file tree
Hide file tree
Showing 15 changed files with 150 additions and 258 deletions.
12 changes: 12 additions & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,18 @@ Unreleased
Breaking API Changes
~~~~~~~~~~~~~~~~~~~~

* scrooge-generator: Dropped the generic (higher-kinded-types) service interface in scala-gen,
users are recommended to use YourService.MethodPerEndpoint, YourService.ServicePerEndpoint
and YourService.ReqRepServicePerEndpoint to represent Thrift service endpoints. Note,
`-finagle` option is required to generated finagle binding code. ``PHAB_ID=D747744``

* scrooge-generator: Removed YourService.FutureIface and YourService[Future] in scala-gen,
use $YourService.MethodPerEndpoint instead. Correspondingly, YourService$FinagleService and
related constructors taking MethodPerEndpoint as parameters. ``PHAB_ID=D747744``

* Scrooge-generator: Dropped ThriftServiceBuilder.build and MethodIfaceBuilder.newMethodIface.
``PHAB_ID=D747744``

* scrooge-generator: Add reserved keywords to ThriftParser. If your field names match
these keywords, you may need to modify them. This change should not affect backwards
and forwards compatiblility if using binary protocol for serde. ``PHAB_ID=D707116``
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,7 @@ import org.apache.thrift.protocol._
class GoldService$FinagleClient(
val service: com.twitter.finagle.Service[ThriftClientRequest, Array[Byte]],
val clientParam: RichClientParam)
extends GoldService.MethodPerEndpoint
with GoldService.FutureIface {
extends GoldService.MethodPerEndpoint {

@deprecated("Use com.twitter.finagle.thrift.RichClientParam", "2017-08-16")
def this(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,14 @@ import org.apache.thrift.transport.TMemoryInputTransport

@javax.annotation.Generated(value = Array("com.twitter.scrooge.Compiler"))
class GoldService$FinagleService(
iface: GoldService[Future],
iface: GoldService.MethodPerEndpoint,
serverParam: RichServerParam
) extends com.twitter.finagle.Service[Array[Byte], Array[Byte]] {
import GoldService._

@deprecated("Use com.twitter.finagle.thrift.RichServerParam", "2017-08-16")
def this(
iface: GoldService[Future],
iface: GoldService.MethodPerEndpoint,
protocolFactory: TProtocolFactory,
stats: StatsReceiver = NullStatsReceiver,
maxThriftBufferSize: Int = Thrift.param.maxThriftBufferSize,
Expand All @@ -39,15 +39,15 @@ class GoldService$FinagleService(

@deprecated("Use com.twitter.finagle.thrift.RichServerParam", "2017-08-16")
def this(
iface: GoldService[Future],
iface: GoldService.MethodPerEndpoint,
protocolFactory: TProtocolFactory,
stats: StatsReceiver,
maxThriftBufferSize: Int
) = this(iface, protocolFactory, stats, maxThriftBufferSize, "GoldService")

@deprecated("Use com.twitter.finagle.thrift.RichServerParam", "2017-08-16")
def this(
iface: GoldService[Future],
iface: GoldService.MethodPerEndpoint,
protocolFactory: TProtocolFactory
) = this(iface, protocolFactory, NullStatsReceiver, Thrift.param.maxThriftBufferSize)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,20 +34,6 @@ import scala.reflect.{ClassTag, classTag}


@javax.annotation.Generated(value = Array("com.twitter.scrooge.Compiler"))
trait GoldService[+MM[_]] extends _root_.com.twitter.finagle.thrift.ThriftService {
/** Hello, I'm a comment. */
def doGreatThings(request: com.twitter.scrooge.test.gold.thriftscala.Request): MM[com.twitter.scrooge.test.gold.thriftscala.Response]

def noExceptionCall(request: com.twitter.scrooge.test.gold.thriftscala.Request): MM[com.twitter.scrooge.test.gold.thriftscala.Response]

/**
* Used to close the underlying `Service`.
* Not a user-defined API.
*/
def asClosable: _root_.com.twitter.util.Closable = _root_.com.twitter.util.Closable.nop
}


object GoldService extends _root_.com.twitter.finagle.thrift.GeneratedThriftService { self =>

val annotations: immutable$Map[String, String] = immutable$Map(
Expand Down Expand Up @@ -1367,12 +1353,16 @@ object GoldService extends _root_.com.twitter.finagle.thrift.GeneratedThriftServ
type noExceptionCall$result = NoExceptionCall.Result


trait MethodPerEndpoint
extends GoldService[Future] {
trait MethodPerEndpoint extends _root_.com.twitter.finagle.thrift.ThriftService {
/** Hello, I'm a comment. */
def doGreatThings(request: com.twitter.scrooge.test.gold.thriftscala.Request): Future[com.twitter.scrooge.test.gold.thriftscala.Response]

def noExceptionCall(request: com.twitter.scrooge.test.gold.thriftscala.Request): Future[com.twitter.scrooge.test.gold.thriftscala.Response]
/**
* Used to close the underlying `Service`.
* Not a user-defined API.
*/
def asClosable: _root_.com.twitter.util.Closable = _root_.com.twitter.util.Closable.nop
}

object MethodPerEndpoint {
Expand Down Expand Up @@ -1425,7 +1415,7 @@ object GoldService extends _root_.com.twitter.finagle.thrift.GeneratedThriftServ

@deprecated("Use MethodPerEndpoint", "2017-11-07")
class MethodIface(serviceIface: BaseServiceIface)
extends FutureIface {
extends MethodPerEndpoint {
def doGreatThings(request: com.twitter.scrooge.test.gold.thriftscala.Request): Future[com.twitter.scrooge.test.gold.thriftscala.Response] =
serviceIface.doGreatThings(self.DoGreatThings.Args(request))
def noExceptionCall(request: com.twitter.scrooge.test.gold.thriftscala.Request): Future[com.twitter.scrooge.test.gold.thriftscala.Response] =
Expand All @@ -1438,41 +1428,16 @@ object GoldService extends _root_.com.twitter.finagle.thrift.GeneratedThriftServ
MethodPerEndpoint(servicePerEndpoint)
}

@deprecated("Use MethodPerEndpointBuilder", "2018-01-12")
implicit object ThriftServiceBuilder
extends _root_.com.twitter.finagle.thrift.service.ThriftServiceBuilder[ServicePerEndpoint, GoldService[Future]] {
def build(servicePerEndpoint: ServicePerEndpoint): MethodPerEndpoint =
MethodPerEndpoint(servicePerEndpoint)
}

implicit object ReqRepMethodPerEndpointBuilder
extends _root_.com.twitter.finagle.thrift.service.ReqRepMethodPerEndpointBuilder[ReqRepServicePerEndpoint, MethodPerEndpoint] {
def methodPerEndpoint(servicePerEndpoint: ReqRepServicePerEndpoint): MethodPerEndpoint =
ReqRepMethodPerEndpoint(servicePerEndpoint)
}

@deprecated("Use MethodPerEndpointBuilder", "2017-11-07")
implicit object MethodIfaceBuilder
extends com.twitter.finagle.thrift.MethodIfaceBuilder[ServiceIface, GoldService[Future]] {
def newMethodIface(serviceIface: ServiceIface): MethodIface =
new MethodIface(serviceIface)
}

@deprecated("Use MethodPerEndpoint", "2017-11-07")
trait FutureIface
extends MethodPerEndpoint
with GoldService[Future] {
/** Hello, I'm a comment. */
def doGreatThings(request: com.twitter.scrooge.test.gold.thriftscala.Request): Future[com.twitter.scrooge.test.gold.thriftscala.Response]

def noExceptionCall(request: com.twitter.scrooge.test.gold.thriftscala.Request): Future[com.twitter.scrooge.test.gold.thriftscala.Response]
}

class FinagledClient(
service: com.twitter.finagle.Service[ThriftClientRequest, Array[Byte]],
clientParam: RichClientParam)
extends GoldService$FinagleClient(service, clientParam)
with FutureIface
with MethodPerEndpoint {

@deprecated("Use com.twitter.finagle.thrift.RichClientParam", "2017-08-16")
Expand Down Expand Up @@ -1509,13 +1474,13 @@ object GoldService extends _root_.com.twitter.finagle.thrift.GeneratedThriftServ
}

class FinagledService(
iface: GoldService[Future],
iface: MethodPerEndpoint,
serverParam: RichServerParam)
extends GoldService$FinagleService(iface, serverParam) {

@deprecated("Use com.twitter.finagle.thrift.RichServerParam", "2017-08-16")
def this(
iface: GoldService[Future],
iface: MethodPerEndpoint,
protocolFactory: org.apache.thrift.protocol.TProtocolFactory,
serviceName: String = "GoldService"
) = this(iface, RichServerParam(protocolFactory, serviceName))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,7 @@ class PlatinumService$FinagleClient(
override val service: com.twitter.finagle.Service[ThriftClientRequest, Array[Byte]],
override val clientParam: RichClientParam)
extends GoldService$FinagleClient(service, clientParam)
with PlatinumService.MethodPerEndpoint
with PlatinumService.FutureIface {
with PlatinumService.MethodPerEndpoint {

@deprecated("Use com.twitter.finagle.thrift.RichClientParam", "2017-08-16")
def this(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,14 @@ import org.apache.thrift.protocol._

@javax.annotation.Generated(value = Array("com.twitter.scrooge.Compiler"))
class PlatinumService$FinagleService(
iface: PlatinumService[Future],
iface: PlatinumService.MethodPerEndpoint,
serverParam: RichServerParam
) extends GoldService$FinagleService(iface, serverParam) {
import PlatinumService._

@deprecated("Use com.twitter.finagle.thrift.RichServerParam", "2017-08-16")
def this(
iface: PlatinumService[Future],
iface: PlatinumService.MethodPerEndpoint,
protocolFactory: TProtocolFactory,
stats: StatsReceiver = NullStatsReceiver,
maxThriftBufferSize: Int = Thrift.param.maxThriftBufferSize,
Expand All @@ -36,15 +36,15 @@ class PlatinumService$FinagleService(

@deprecated("Use com.twitter.finagle.thrift.RichServerParam", "2017-08-16")
def this(
iface: PlatinumService[Future],
iface: PlatinumService.MethodPerEndpoint,
protocolFactory: TProtocolFactory,
stats: StatsReceiver,
maxThriftBufferSize: Int
) = this(iface, protocolFactory, stats, maxThriftBufferSize, "PlatinumService")

@deprecated("Use com.twitter.finagle.thrift.RichServerParam", "2017-08-16")
def this(
iface: PlatinumService[Future],
iface: PlatinumService.MethodPerEndpoint,
protocolFactory: TProtocolFactory
) = this(iface, protocolFactory, NullStatsReceiver, Thrift.param.maxThriftBufferSize)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,18 +33,6 @@ import scala.reflect.{ClassTag, classTag}


@javax.annotation.Generated(value = Array("com.twitter.scrooge.Compiler"))
trait PlatinumService[+MM[_]] extends GoldService[MM] {

def moreCoolThings(request: com.twitter.scrooge.test.gold.thriftscala.Request): MM[Int]

/**
* Used to close the underlying `Service`.
* Not a user-defined API.
*/
override def asClosable: _root_.com.twitter.util.Closable = _root_.com.twitter.util.Closable.nop
}


object PlatinumService extends _root_.com.twitter.finagle.thrift.GeneratedThriftService { self =>

val annotations: immutable$Map[String, String] = immutable$Map.empty
Expand Down Expand Up @@ -936,11 +924,14 @@ object PlatinumService extends _root_.com.twitter.finagle.thrift.GeneratedThrift
type moreCoolThings$result = MoreCoolThings.Result


trait MethodPerEndpoint
extends com.twitter.scrooge.test.gold.thriftscala.GoldService.MethodPerEndpoint
with PlatinumService[Future] {
trait MethodPerEndpoint extends com.twitter.scrooge.test.gold.thriftscala.GoldService.MethodPerEndpoint {

def moreCoolThings(request: com.twitter.scrooge.test.gold.thriftscala.Request): Future[Int]
/**
* Used to close the underlying `Service`.
* Not a user-defined API.
*/
override def asClosable: _root_.com.twitter.util.Closable = _root_.com.twitter.util.Closable.nop
}

object MethodPerEndpoint {
Expand Down Expand Up @@ -989,7 +980,7 @@ object PlatinumService extends _root_.com.twitter.finagle.thrift.GeneratedThrift
@deprecated("Use MethodPerEndpoint", "2017-11-07")
class MethodIface(serviceIface: BaseServiceIface)
extends com.twitter.scrooge.test.gold.thriftscala.GoldService.MethodIface(serviceIface)
with FutureIface {
with MethodPerEndpoint {
def moreCoolThings(request: com.twitter.scrooge.test.gold.thriftscala.Request): Future[Int] =
serviceIface.moreCoolThings(self.MoreCoolThings.Args(request))
}
Expand All @@ -1000,40 +991,16 @@ object PlatinumService extends _root_.com.twitter.finagle.thrift.GeneratedThrift
MethodPerEndpoint(servicePerEndpoint)
}

@deprecated("Use MethodPerEndpointBuilder", "2018-01-12")
implicit object ThriftServiceBuilder
extends _root_.com.twitter.finagle.thrift.service.ThriftServiceBuilder[ServicePerEndpoint, PlatinumService[Future]] {
def build(servicePerEndpoint: ServicePerEndpoint): MethodPerEndpoint =
MethodPerEndpoint(servicePerEndpoint)
}

implicit object ReqRepMethodPerEndpointBuilder
extends _root_.com.twitter.finagle.thrift.service.ReqRepMethodPerEndpointBuilder[ReqRepServicePerEndpoint, MethodPerEndpoint] {
def methodPerEndpoint(servicePerEndpoint: ReqRepServicePerEndpoint): MethodPerEndpoint =
ReqRepMethodPerEndpoint(servicePerEndpoint)
}

@deprecated("Use MethodPerEndpointBuilder", "2017-11-07")
implicit object MethodIfaceBuilder
extends com.twitter.finagle.thrift.MethodIfaceBuilder[ServiceIface, PlatinumService[Future]] {
def newMethodIface(serviceIface: ServiceIface): MethodIface =
new MethodIface(serviceIface)
}

@deprecated("Use MethodPerEndpoint", "2017-11-07")
trait FutureIface
extends com.twitter.scrooge.test.gold.thriftscala.GoldService.FutureIface
with MethodPerEndpoint
with PlatinumService[Future] {

def moreCoolThings(request: com.twitter.scrooge.test.gold.thriftscala.Request): Future[Int]
}

class FinagledClient(
service: com.twitter.finagle.Service[ThriftClientRequest, Array[Byte]],
clientParam: RichClientParam)
extends PlatinumService$FinagleClient(service, clientParam)
with FutureIface
with MethodPerEndpoint {

@deprecated("Use com.twitter.finagle.thrift.RichClientParam", "2017-08-16")
Expand Down Expand Up @@ -1070,13 +1037,13 @@ object PlatinumService extends _root_.com.twitter.finagle.thrift.GeneratedThrift
}

class FinagledService(
iface: PlatinumService[Future],
iface: MethodPerEndpoint,
serverParam: RichServerParam)
extends PlatinumService$FinagleService(iface, serverParam) {

@deprecated("Use com.twitter.finagle.thrift.RichServerParam", "2017-08-16")
def this(
iface: PlatinumService[Future],
iface: MethodPerEndpoint,
protocolFactory: org.apache.thrift.protocol.TProtocolFactory,
serviceName: String = "PlatinumService"
) = this(iface, RichServerParam(protocolFactory, serviceName))
Expand Down
Original file line number Diff line number Diff line change
@@ -1,19 +1,22 @@
package com.twitter.scrooge.backend

import com.twitter.conversions.DurationOps._
import com.twitter.scrooge.testutil.Spec
import com.twitter.util.Await
import com.twitter.util.Future

class NamespaceSpec extends Spec {
"Scala Generator" should {
import bar._
import com.fake._
"import from another namespace" in {
val service: Restaurant[Some] = new Restaurant[Some] {
def isOpen(whichDay: Weekday) = Some(whichDay != Weekday.Monday)
val service: Restaurant.MethodPerEndpoint = new Restaurant.MethodPerEndpoint {
def isOpen(whichDay: Weekday) = Future.value(whichDay != Weekday.Monday)
def makeReservation(whichDay: Weekday, howMany: Int) =
Some(if (whichDay == Weekday.Monday) 0 else howMany)
Future.value(if (whichDay == Weekday.Monday) 0 else howMany)
}
service.makeReservation(Weekday.Monday, 2) must be(Some(0))
service.makeReservation(Weekday.Tuesday, 2) must be(Some(2))
Await.result(service.makeReservation(Weekday.Monday, 2), 5.seconds) must equal(0)
Await.result(service.makeReservation(Weekday.Tuesday, 2), 5.seconds) must equal(2)
}
}
}

This file was deleted.

3 comments on commit 8d768ca

@henryxparker
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sad to see HKTs go.
Twitter futures are not an ideal interface for writing code in a functional style, and the HKT served as an easy way to switch that interface.
There is even a library for it.
I do not look forward to the boilerplate that'll be required if we need to upgrade.

Perhaps this use-case is a vocal minority, and others appreciate the more compact generated code.
Nevertheless I think it is worth reconsidering dropping the HKT's.

@mosesn
Copy link
Contributor

@mosesn mosesn commented on 8d768ca Jan 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@henryxparker thanks for the feedback! We actually didn't realize that anyone was using this interface. I had never seen the dwolla library before. As far as we knew it was an unused API (we have never used it internally).

Can you tell me more about how you use Scrooge? Do you use Scrooge + Finagle under the hood, but convert all of the futures to Scala Futures with a bijection?

Also, would you mind filing an issue related to this? I'm not sure if we'll reintroduce HKT into the normal Scrooge generated code, but we may be able to continue generating it as a plugin.

I'm sorry for breaking your use case, but I'm optimistic that we can work out a compromise that will work for you.

@henryxparker
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mossen thank you so much for the response! I do indeed use scrooge with finagle.
And it's not exactly a bijection. I have access to a twitterFuture.asScala extension method and vice-versa that can take care of that conversion with relative ease, so that's not the primary reason I use the HKT interface.

I use the HKT interface because I started using cats.effect.IO. The reasoning being the same as in this article.
Since Futures start executing as soon as they are defined, I could no longer use the twitterFuture.asIO extension method syntax if I wanted to keep the advantages of IO.
That's why the dwolla library exists. It provides a way to automatically convert a Service[Future] to a Service[IO] safely.

I've added the issue #352 as requested. Thanks again for listening!

Please sign in to comment.